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About  Tffls  Report 

This  report  is  one  of  ten  individual  studies  conducted  for  the  Flathead  Basin  Forest  Practices/ 
Water  Quality  and  Fisheries  Cooperative  Program.  The  Cooperative  Program  was  administered  by 
a  Coordinating  Team  representing  the  Montana  Department  of  State  Lands  Forestry  Division,  the 
Flathead  National  Forest,  Plum  Creek  Timber  Company,  L.P.,  the  Montana  Department  of  Fish, 
Wildlife  and  Parks,  the  Montana  Department  of  Health  and  Environmental  Sciences'  Water  Quality 
Bureau,  the  University  of  Montana,  and  the  Flathead  Basin  Commission. 

The  Cooperative  Program's  specific  objectives  were  (1)  to  document,  evaluate,  and  monitor 
whether  forest  practices  affect  water  quality  and  fisheries  within  the  Flathead  Basin,  and  (2)  if 
detrimental  impacts  exist,  to  establish  a  process  to  utilize  this  information  to  develop  criteria  and 
administrative  procedures  for  protecting  water  quality  and  fisheries. 

The  ten  individual  studies  included  the  evaluation  of:  (1)  specific  practices  at  the  site  level,  (2) 
accumulation  of  practices  at  the  watershed  level,  (3)  general  stream  conditions,  (4)  water  quality 
variables  relative  to  levels  of  management  activity  in  small  watersheds,  (5)  fish  habitat  and 
abundance  relative  to  stream  variables  influenced  by  forest  practices  at  the  watershed  level,  (6)  long- 
term  changes  in  large-stream  dynamics  related  to  historical  records  of  natural  and  man-related 
disturbances,  and  (7)  changes  in  lake  sediments  relative  to  historical  records  of  natural  and  man- 
related  disturbances.  A  Final  Report  was  developed  which  contains  summaries  of  each  of  the  studies, 
a  set  of  summary  conclusions  and  recommendations,  and  a  formal  response  to  the  recommendations 
by  the  land  management  organizations  which  administered  the  Cooperative  Program. 
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EXECUTIVE  SUMMARY 

In  1988,  the  Flathead  Basin  Forest  Practices/Water 
Quality  and  Fisheries  Cooperative  was  formed  to  address 
questions  and  concerns  surrounding  forest  management  and 
water  quality  in  the  Flathead  Basin.   The  Cooperative  has 
funded  a  series  of  research  projects  in  the  Flathead  Basin 
to  assess  cause-and-ef feet  relationships  among  forest 
practices,  water  quality  and  fisheries.   In  particular,  the 
projects  are  concerned  with  identifying  cumulative  watershed 
effects. 

One  of  these  projects  used  a  forest  practice  monitoring 
tool  which  involves  on-site  assessments  of  Best  Management 
Practices  (BMPs) .   The  BMP  assessment  was  intended  to 
measure  the  level  of  practice  application  and  practice 
effectiveness  in  preventing  soil  erosion  and  protecting 
water  quality  in  the  Flathead  Basin. 

The  BMP  assessment  had  two  objectives:   1)  The  results 
would  document  the  level  of  application  and  skill  in 
implementation  of  the  State's  voluntary  Best  Management 
Practices,  and  their  effectiveness  in  controlling  nonpoint 
source  pollution,  and  2)  A  comparison  would  be  made  between 
these  results  and  those  of  the  1988  Montana  Legislature's 
Environmental  Quality  Council  (EQC)  statewide  BMP 
assessment. 

This  report  includes  a  thorough  examination  of  the 
methodology  of  BMP  assessment.   The  various  aspects  of 


setting  up  the  study  and  conducting  the  field  work  are 
discussed.   This  information  is  summarized  to  document 
procedures  and  shed  light  on  possible  alternatives  and 
suggestions  for  improving  similar  studies  in  the  future. 
In  order  to  provide  a  basis  for  comparison  with  the 
work  of  the  EQC,  similar  site  selection  criteria  were 
utilized  in  this  study.   A  stratified  random  sample  of  52 
timber  sales  were  selected  from  4  land  ownership  groups: 
Flathead  National  Forest  -  22  sales;  Plum  Creek  Timber 
Company  -  19  sales;  Montana  Department  of  State  Lands  -  6 
sales;  and  Non-Industrial  Private  -  5  sales.   All  timber 
sales  were  harvested  between  January  1986  and  December  1988. 
Each  sale  was  evaluated  on  up  to  39  separate  practices. 
These  practices  corresponded  to  the  Best  Management 
Practices  for  Forestry  (December  1988  revision)  developed  by 
the  Environmental  Quality  Council's  BMP  Technical  Committee. 
Ratings  were  carried  out  by  3  teams,  each  composed  of  5 
members  representing  industry,  state  and  federal  agencies, 
and  environmental  groups.   Each  team  member  had  technical 
expertise  in  some  aspect  of  forest  or  watershed  management. 

The  Flathead  Basin  timber  sale  audits  revealed  that  90 
percent  of  all  management  practices  were  adequately  applied; 
7  percent  of  the  practices  were  rated  "minor"  departures; 
and  3  percent  were  rated  as  "major"  departures.   Generally 
there  was  a  close  correlation  between  the  failure  to 


adequately  apply  a  BMP  and  the  resulting  impact  which  was 
observed.   Major  BMP  departures  produced  major  resource 
impacts . 

In  8  of  the  52  sales  (15  percent),  audit  teams 
identified  at  least  one  practice  as  having  "major 
detrimental  impacts  on  soil  and  water  resources."   Impacts 
were  considered  "extensive  and  long  term"  in  2  of  these 
sales.   The  "major"  impacts  observed  on  the  other  6  sales 
were  considered  to  impart  short  term  effects. 

In  31  of  the  52  sales  (59  percent),  audit  teams 
identified  at  least  1  practice  as  having  "minor  detrimental 
impacts  on  soil  and  water  resources".   Minor  impacts  were 
all  considered  to  be  of  short  duration. 

The  impacts  have  been  summarized  by  the  major 
categories  of  BMPs  identified  on  the  audit  form.   The 
"Timber  Harvest"  category,  which  represents  11  of  the  39 
practices  audited,  contributed  the  greatest  number  of 
potential  impacts  to  water  quality  -  more  than  3  0  percent  of 
all  impacts,  both  "major"  and  "minor",  came  from  this  one 
category  alone.   Most  of  these  deficiencies  were  associated 
with  practices  in  the  streamside  management  zones  (SMZs)  and 
with  the  location  and  drainage  of  skid  trails. 

The  "Road  Drainage"  category  involved  5  of  the  39 
practices  audited,  but  accounted  for  over  2  0  percent  of  all 
"major"  and  "minor"  impacts  observed. 


The  Montana  Department  of  State  Lands1  timber  sales 
were  the  only  ones  on  which  major  impacts  were  not  observed. 
Major  impacts  were  most  frequently  found  on  the  Private 
Non-Industrial  sales,  and  40  percent  of  their  sales  had  at 
least  one  major  impact.   The  Flathead  National  Forest  and 
Plum  Creek  Timber  Co.  rated  very  similarly  with 
approximately  14  percent  and  16  percent  of  their  sales, 
respectively,  registering  at  least  1  major  impact. 

This  assessment  included  the  same  set  of  BMPs  that  were 
audited  in  the  1988  EQC  field  study,  as  well  as  additional 
or  amended  practices  added  to  the  audit  form  as  a  result  of 
adopted  changes  in  the  BMPs  resulting  from  the  HJR  49 
report.   Most  of  those  changes  involved  improvements  of 
practices  associated  with  streamside  management  zones, 
including  a  minimum  SMZ  width  criteria.   This  practice  was 
evaluated  to  determine  its  present  level  of  application  in 
the  Flathead  Basin,  but  was  not  included  in  the  comparative 
analysis  with  the  EQC  results. 

Overall,  the  ratings  indicate  that  BMPs  are  being 
applied  and  are  effective  in  a  relatively  high  percentage  of 
timber  harvesting  operations  in  the  Flathead  Basin.    Care 
must  be  exercised  in  interpreting  results  based  solely  upon 
the  percentage  of  the  practices  rated  as  "adequately 
applied."   Extensive  erosion  was  observed  on  a  number  of 
sales  and  it  resulted  from  only  a  few  inadequately  applied 
practices.   Every  practice  evaluated  did  not  have  the  same 


potential  to  protect  water  quality  from  nonpoint  source 
pollution. 

Comparison  of  these  results  with  those  from  the  EQC 
study  indicate  that  BMPs  are  better  applied  and  are 
generally  more  effective  in  the  Flathead  Basin  than  they  are 
statewide. 

However,  there  is  clearly  room  for  improvement.  It  was 
found  that  1  "major"  (either  short-  or  long-term)  impact  to 
water  quality  is  occurring  for  every  2  timber  sales 
conducted  in  the  Flathead.   It  was  also  found  that  almost  2 
"minor"  (short-  term)  impacts  per  timber  sale  were 
occurring. 

The  damage  to  a  watershed  resulting  from  a  single 
timber  sale  with  major  practice  departures  and  impacts  is 
generally  easy  to  recognize.   However,  minor  impacts  should 
not  be  underestimated  and  are  a  concern  from  a  cumulative 
effects  perspective.   The  primary  benefit  from  an  assessment 
process  like  a  BMP  audit  is  to  reduce  the  potential  risk  of 
cumulative  watershed  effects.   Discovering  and  controlling 
minor  practice  departures  and  impacts  from  various  timber 
sales  across  mixed  ownerships,  may  decrease  the  potential 
for  basin-wide  effects. 

The  highest  scores  (best  BMP  implementation)  awarded 
were  frequently  on  sites  where  harvest  boundaries  were 
adjacent  to  perennial  streams  that  had  clearly  defined  banks 


and  channels  and  easily  recognized  beneficial  uses  (eg. 
trout  habitat) . 

Often  the  lowest  scores  given  were  on  sites  that  would 
not  have  been  expected  to  produce  potential  water  quality 
problems.   This  illustrates  the  need  for  a  program  of 
continued  education  to  aid  in  interpretation  and  selection 
of  correct  practices  under  a  variety  of  site  conditions. 

Timber  sales  on  which  departures  and  impacts  were  more 
frequently  observed  were  characterized  by  the  following: 

*  Non-Industrial  Private  ownership.   This  ownership 
class  registered  the  poorest  performance  in  BMP 
application  and  effectiveness.   Furthermore,  the 
group  was  the  least  cooperative  participant  in  the 
study,  frequently  denying  us  access  to  their 
lands. 

*  Large  management  areas,  where  uniform  practices 
were  applied  and  not  tailored  to  micro-site 
conditions 

*  Higher-elevation  headwaters  drainages  with  poorly 
defined  stream  channels 

*  Wet  or  moist  sites  with  either  shallow  water 
tables  and/or  high  stream  drainage  densities. 

*  Older  sales  where  planning  of  transportation 
systems  and  sale  layout  pre-dated  the  recognition 
of  statewide  BMPs. 

*  Sales  which  did  not  physically  mark  or  delineate 


streamside  management  zones. 

*  Inadequate  road  drainage  features  on  active  system 
roads  as  well  inadequate  drainage  on  roads  that 
have  been  closed. 

*  Sales  which  lacked  a  routine  maintenance 
schedule  for  ditches,  culverts,  and  road  surfaces. 

*  Sales  which  were  conducted  on  highly  erodible 
soils,  where  the  increased  risk  of  sediment 
production  requires  an  adjustment  in  the  frequency 
and  standards  of  erosion  preventative  measures. 

Other  findings  and  recommendations: 

*  No  formalized  process  to  conduct  BMP  assessments 
exists,  although  assessments  are  periodically 
conducted  on  the  Department  of  State  Lands  by 
their  hydrologist  and  soil  scientist. 

*  Refinements  in  the  BMP  audit  process  are  needed 
to  remove  the  subjective  nature  of  the  process 
and  tailor  the  rating  scales  to  more  specifically 
address  nonpoint  source  pollution  and  sediment 
delivery. 

*  Efforts  at  educating  loggers,  equipment  operators 
and  sale  administrators  have  been  initiated  by 
the  Montana  Logging  Association  and  Plum  Creek 
Timber  Company's  Kalispell  management  unit.   A 
comprehensive  cooperative  program  of  education, 
either  through  instructional  tapes  or  training 


sessions  is  needed.  The  goal  of  the  education 
effort  would  be  to  reach  a  targeted  number  of 
operators,  administrators  and  sale  planners 
throughout  the  Basin. 

*  Improve  the  communication  and  flexibility  between 
the  sale  planning  and  sale  administration  on 
U.S.F.S. .This  would  allow  for  the  administrator 
to  adjust  sale  boundaries  and  harvesting  systems 
for  improved  BMP  compliance. 

*        Improve  communication  between  the  sale 

administrators  and  equipment  operators  to 
identify  areas  of  sales,  including  the  SMZs, 
which  require  special  treatment  or  specialized 
best  management  practices. 

«    Implement  clear  delineation  of  SMZs  to  aid 
equipment  operators  in  recognizing  them. 

'        Increase  on-site  interaction  between  sale 

administrators  and  soil  and  water  specialists. 
This  would  allow  for  more  complete  review  of  site 
conditions  and  adjustment  of  practices  for  better 
resource  protection. 

*  Adopt  targets  for  achieving  BMP  application  and 
effectiveness  goals  for  all  forest  management 
activities  in  the  basin.   Develop  a  consistent 
procedure  for  future  BMP  audits  to  monitor 
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progress  toward  reaching  these  goals. 
*   BMP  assessments,  by  themselves,  are  unsuited  for 
quantification  of  any  cumulative  watershed 
impact,  but  do  address  this  problem  through  the 
fundamental  assumption  that  if  BMPs  are  properly 
applied  and  effective,  then  cumulative  watershed 
effects  may  be  minimized. 
This  study  provided  the  first  opportunity  for  many 
Flathead  Basin  resource  professionals  from  both  timber 
industry  and  land  management  agencies  to  participate  in  BMP 
assessment.   The  feedback  we  received  from  many  participants 
indicates  that  much  of  the  value  of  this  effort  was  in 
education.    The  active  participation  in  the  field  reviews 
and  the  interest  of  many  individuals  in  the  Flathead  Basin 
is  important  to  the  continued  success  of  such  efforts.   Only 
with  support  and  encouragement  from  the  upper  levels  of 
management  will  programs  such  as  this  nonpoint  source 
monitoring  project  work. 

INTRODUCTION 
Management  of  forest  lands  for  timber  and  non-timber 
resources  has  raised  a  host  of  complex  issues.   Protecting 
non-timber  resources  during  timber  harvest  requires  managers 
to  carefully  plan  and  execute  their  activities.   This 
requires  knowledge  of  how  the  land  will  respond  to  various 
activities  and  a  means  of  monitoring  to  verify  protection 
and  provide  feedback  to  guide  us  in  future  management. 
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Concerns  expressed  by  both  resource  managers  and  the 
public  over  the  impacts  of  timber  harvesting  on  non-timber 
resources,  such  as  fisheries  and  water  guality,  cause  us  to 
look  for  more  accurate  and  efficient  means  to  achieve 
resource  protection. 

In  the  Flathead  River  Basin,  timber  industry  and 
federal  and  state  land  management  agencies  are  facing  timber 
supply  problems  because  of  appeals  and  injunctions 
reflecting  public  concern  for  non-timber  resources.   To 
address  this,  the  Flathead  Basin  Water  Quality  and  Fisheries 
Cooperative  was  formed  to  try  to  answer  the  basic  guestion 
"are  forest  practices  affecting  fisheries  and  water 
guality?"   This  is  a  guestion  that  can  be  answered  with 
extensive  monitoring. 

Unfortunately,  monitoring  complex  relationships  in 
large  watersheds  is  not  well-served  by  traditional 
guantitative  technigues.   The  optimal  method  should 
demonstrate  direct  cause  and  effect  relationships  over  a 
large  geographical  areas  such  as  the  Flathead  River  Basin, 
address  the  complexities  of  mixed  ownership  and  differing 
management  objectives  in  watersheds,  allow  for  distinction 
between  man-caused  changes  and  natural  variation  in  water 
guality,  and  yield  results  in  a  relatively  short  time.   The 
optimal  method  does  not  exist. 

One  possible  starting  point,  however,  is  a  subjective, 
gualitative   approach  utilizing  interdisciplinary  teams  that 
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rate  a  series  of  forest  practices  and  their  potential  to 
produce  sediment,  thereby  impacting  water  quality.   Best 
Management  Practices  (BMP)  assessment  is  a  method  that  fs 
compatible  with  the  concerns  identified  above  while 
addressing  the  issues  of  non-point  pollution  and  "cumulative 
watershed  effects"  for  which  federal  legislation  mandates 
identification  and  control. 

In  1988  the  first  statewide  BMP  assessment  was 
conducted  by  the  Montana  Environmental  Quality  Council 
(EQC) .   The  effort  was  recognized  as  an  excellent  progress 
report  on  implementation  of  voluntary  BMPs  and  their 
effectiveness  in  protecting  soil  and  water  resources. 
Recommendations  resulting  from  the  EQC  study  included:  1) 
continued  education  to  improve  the  level  of  compliance  and 
2)  further  assessments  as  a  means  of  monitoring  progress. 

A  primary  objective  of  this  study  was  to  intensively 
replicate   that  effort  in  the  Flathead  River  Basin,  and 
compare  the  results  obtained  with  those  from  the  state-wide 
study . 

Several  states  throughout  the  country  are  presently 
using  this  method  of  monitoring  as  a  tool  to  measure 
compliance  with  their  Forest  Practices  Acts.   Montana's 
non-regulatory  atmosphere  has  relied  on  a  cooperative 
approach  with  a  voluntary  set  of  practices.   Assessments  of 
this  kind  in  Montana  are  not  designed  to  be  punitive  in 
nature.   Instead  they  are  intended  to  serve  4  purposes:  to 
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provide  the  resource  manager  with  necessary  feedback  on 
implementation  of  the  practices  across  their  ownership;  to 
improve  the  clarity  of  intent  of  a  practice;  to  indicates 
areas  in  need  of  increased  education;  and  to  measure 
progress  and  performance  in  obtaining  goals  or  objectives 
associated  with  natural  resource  management. 

The  success  of  a  cooperative  monitoring  program  which 
employs  an  interdisciplinary  approach,  rests  heavily  upon 
the  spirit  and  attitude  of  the  participants.   The  ability  of 
individuals  to  reach  agreement  on  forest  practice 
application,  as  professionals  and  in  an  objective  manner,  is 
crucial  to  the  success  of  the  project. 

The  atmosphere  of  these  field  reviews  can  be 
confrontational  at  times.   It  is  not  an  easy  task  to 
confront  individuals  with  criticism  of  their  work.   Nor  is 
it  always  easy  for  some  individuals  to  accept  that  criticism 
due  to  anxiety  over  possible  repercussions  from  it. 
Reprimands  from  supervisors,  peer-pressure,  possible 
accusations  of  resource  mismanagement,  or  concerns  over  the 
potential  misuse  of  the  data  by  the  press  or  public,  all 
loom  as  obstacles  to  achieving  our  objective.   That 
objective  is  to  carry  out  a  factual  assessment  of  the 
impacts  on  water  guality  resulting  from  forest  practices  in 
the  Flathead  River  Basin.   Attention  should  be  focused  on 
the  positive  aspects  and  knowledge  which  these  studies 
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contribute  toward  improving  the  future  management  of  natural 
resources. 

This  study  provides  only  one  piece  of  the  puzzle 
assessing  nonpoint  source  pollution  from  forest  practices. 
Further  research  is  presently  being  conducted  which  will  add 
to  our  knowledge  of  this  complex  issue. 

BMP  ASSESSMENTS  FOR  MONITORING  WATER  QUALITY 

The  Federal  Water  Pollution  Control  Act  Amendments  of 
1972  (U.S.  Congress,  1972),  later  renamed  the  Clean  Water 
Act,  mandated  in  Section  208  that  States  develop  an  "area- 
wide  waste  treatment  management  planning  process"  that  must 
include  "a  process  to  (i)  identify. . .silviculturally  related 
nonpoint  sources  of  pollution  . . .  and  their  cumulative 
effects  ...  and  (ii)  set  forth  procedures  and  methods  to 
control  ...  such  sources."  While  resource  managers  agree 
"nonpoint  sources"  and  "cumulative  effects"  are  real  and 
important  contributors  to  water  quality  problems,  they  have 
yet  to  agree  on  a  method  to  isolate  and  quantitatively 
measure  them.  Monitoring  watersheds  to  assess  the 
effectiveness  of  nonpoint  source  control  programs  and 
answering  questions  regarding  the  impact  of  forest  practices 
on  water  quality  poses  a  unique  set  of  problems.   These 
problems  will  become  more  evident  upon  examining  the 
following  definitions. 

Nonpoint  source  pollution  resulting  from  silvicultural 
practices  is  a  change  in  water  quality  that  exceeds 
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"natural"  conditions,  comes  from  a  multitude  of  locations, 
and  has  no  identifiable  discharge  point  for  applying  control 
measures. 

Cumulative  effects  are  changes  in  water  guality, 
streamflow,  channel  structure,  or  aguatic  habitat  caused  by 
the  interaction  of  natural  ecosystem  processes  with  multiple 
forest  practice  operations  (EQC,  1988) . 

One  problem  with  both  of  these  definitions  is  that  they 
contain  a  natural  component.   For  states  to  identify  and 
control  nonpoint  sources  and  cumulative  effects,  a  method  is 
needed  to  identify  naturally  occurring  sediment  production 
and  separate  it  from  sediment  production  resulting  from 
forest  practices. 

Identifying  natural  water  guality,  under 
pre-disturbance  conditions  is  extremely  complex  due  to  the 
typically  large  natural  variability  in  water  guality.   This 
is  due  to  the  interaction  of  geologic  materials  and  soils, 
vegetation,  climatic  events,  landforms  and  land  use  history. 
Sampling,  in  an  attempt  to  capture  this  variability, 
reguires  sophisticated  and  costly  baseline  water  guality 
monitoring  programs.   Large  numbers  of  samples  must  be 
collected  at  the  right  times  to  accurately  characterize 
water  guality,  and  sampling  should  be  conducted  over  a  long 
period  to  capture  the  range  of  climatic  events.   The  cost  of 
a  single  sediment  monitoring  station  can  be  up  to  $30,000 
per  year  (NCASI,  1988).   At  these  costs  few  streams  can  be 
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adequately  monitored,  resulting  in  a  lack  of  available 
baseline  data.   Without  baseline  data,  legislation  is 
difficult  to  enforce,  unless  operators  are  caught  in  the 
act. 

The  fact  that  many  watersheds  are  a  mosaic  of  mixed 
ownerships  further  complicates  efforts  to  account  for, 
identify  and  control  nonpoint  sources  and  cumulative 
effects.   A  cumulative  effect  in  these  watersheds  is  the 
result  of  multiple  land  owners  contributing  only 
incrementally  to  sediment  production.   The  amounts  may  be 
insignificant  when  considered  alone,  but  in  combination  with 
several  other  sites  can  be  significant.   The  effects  may  not 
be  evident  for  a  considerable  distance  downstream.   This 
makes  mixed-ownerships  particularly  difficult  to  assess 
using  traditional  sampling  methods.   Problems  include  a  lack 
of  identifiable  pollutant  discharge  points  and  a  lack  of 
baseline  data.   Perhaps  the  biggest  problem,  however,  is 
obtaining  a  simple  inventory  of  all  past  activities  in 
large,  mixed-ownership  watersheds.  The  inventory  would  aid 
in  prioritizing  monitoring  efforts.   Consequently,  agencies 
have  looked  for  acceptable  substitutes  for  direct 
measurements  of  nonpoint  source  pollution. 

One  such  acceptable  substitute  for  water  quality 
monitoring  is  a  survey  of  compliance  with  Best  Management 
Practices  (BMPs) .  BMPs  are  developed  through  methods  which 
draw  heavily  upon  the  "best  professional  judgement"  of 
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resource  professionals  in  the  region.   They  are  determined 
to  be  the  most  effective,  practical  means  of  preventing  or 
reducing  the  amount  of  pollution  generated  by  a  nonpoint 
source  to  a  level  compatible  with  water  guality  goals. 

BMP  monitoring  can  not  guantify  cumulative  watershed 
effects.   But,  based  on  the  assumption  that  if  BMPs  are 
adeguately  applied,  and  they  are  effective  in  controlling 
erosion,  then  the  likelihood  of  contributing  to  a  cumulative 
effect  will  be  minimized. 

Federal  legislative  support  for  this  approach  to 
monitoring  can  be  found  in  Section  319  of  the  Clean  Water 
Act  of  1987.  It  reguires  states  to  "...identify  BMPs  which 
will  be  undertaken  to  reduce  pollutant  loadings  resulting 
from  nonpoint  source  pollution,  . . .  and  assess  the 
effectiveness  of  their  silvicultural  nonpoint  source  control 
programs."   Since  BMPs  are  recognized  by  state  and  federal 
legislation  as  the  method  to  reduce  nonpoint  source 
pollution,  it  is  appropriate  to  incorporate  an  assessment  of 
BMP  compliance  into  a  nonpoint  control  program.    States  are 
increasingly  relying  on  gualitative  surveys  consisting  of 
on-site  assessments  of  BMPs  by  interdisciplinary  teams  to 
assess  their  silvicultural  nonpoint  source  control  programs 
(NCASI, 1988) .   States  such  as  California,  Idaho,  Oregon, 
Utah  and  Washington  rely  on  a  gualitative  approach  to 
assessing  nonpoint  control. 
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Recognizing  the  problems  associated  with  applying  water 
guality  standards  to  nonpoint  source  activities,  the  EPA  has 
supported  the  use  of  BMP  assessment  as  a  substitute  for 
direct  monitoring,  stating  that  "...once  BMPs  have  been 
approved  by  the  state,  the  BMPs  become  the  primary  mechanism 
for  meeting  water  guality  standards"  (Jensen,  1987)  . 

In  April  of  1987,  the  Montana  Cumulative  Watershed 
Effects  Cooperative,  consisting  of  federal  and  state 
government  agencies  and  private  industry,  signed  a  "Memo  of 
Understanding  for  Adopting  and  Implementing  Best  Management 
Practices  for  Forestry  in  Montana."   This  was  the  first 
attempt  to  identify  a  set  of  BMPs  statewide.   During 
Montana's  1987  legislative  session,  the  House  Joint 
Resolution  49  directed  the  Environmental  Quality  Council 
(EQC)  to  study  "how  current  forest  management  practices  are 
affecting  watersheds  in  Montana,  the  range  of  effective 
management  practices  to  conserve  watersheds,  the  existing 
administrative  framework  promoting  the  use  of  BMPs,  and 
areas  of  potential  improvement"  (EQC,  1988)  . 

A  product  of  the  EQC  study  of  1988  was  a  revised  set  of 
BMPs  for  the  state  that  incorporated  input  from  a  wide  range 
of  resource  professionals.   In  addition,  they  cited  the 
Forestry  Division  of  the  Department  of  State  Lands  as  the 
lead  agency  in  conducting  future  audits.    Since  then  the 
BMPs  have  gone  through  yet  another  revision  and  were 
recently  approved  by  the  Environmental  Protection  Agency  as 
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part  of  the  State  of  Montana's  Nonpoint  Source  Pollution 
Control  Program. 

Although  BMP  assessment  has  a  number  of  advantages 
which  have  made  it  a  popular  choice,  it  is  not  without 
limitations.   The  wording  of  BMPs  is  designed  to  provide 
some  flexibility  of  interpretation  by  resource  managers, 
conseguently  many  of  the  practices  lack  a  clearly  defined 
intent.   This  produces  difficulty  in  obtaining  a  consensus 
on  what  constitutes  an  adequate  practice.   During  field 
assessment,  BMPs  should  be  evaluated  based  on  the  collective 
professional  judgement  of  the  team  members  with  respect  to 
site-specific  needs  and  the  skill  of  application  of  the 
practice.   BMPs  should  not  be  interpreted  to  include 
additional  meanings  or  added  levels  of  protection.   Areas 
where  consensus  is  difficult  to  obtain  should  be  viewed  as 
possible  areas  for  improvement  in  the  clarity  of  the  BMP 
language. 

It  is  probably  accurate  to  assume  that  if  difficulty  is 
encountered  by  resource  specialists  in  the  interpretation  of 
BMPs,  then  opportunities  for  misinterpretation  by  operators 
on  the  ground  will  be  equally  likely. 

BMPs  alone  are  not  intended  to  or  capable  of  adequately 
addressing  all  water  quality  concerns  associated  with  forest 
practices.   Assessments  to  date  have  only  addressed  one 
contributor  to  the  cumulative  watershed  effects  issue  - 
surface  erosion.   Another  component,  in-channel  sediment 
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production  resulting  from  the  channel  scour,  is  left 
unaddressed  by  this  process.   This  latter  component  can  be  a 
significant  contributor  to  increased  sedimentation  in 
streams.   It  results  from  the  increasing  water  yields  and 
the  changing  of  the  timing  of  hydrologic  events  associated 
with  removing  the  vegetative  canopy  from  a  site. 
BMP  assessments  don't  produce  finite  answers  about  sediment 
production  resulting  from  forest  management  practices. 
Instead  the  data  are  qualitative  in  nature.   The  data  can  be 
used  to  make  broad  statements  of  practice  compliance  and 
effectiveness  while  pointing  to  areas  in  need  of 
improvement . 

The  BMP  assessment  process  is  in  its  infancy  in 
Montana.  This  audit  of  Flathead  Basin  practices  is  only  the 
second  time  an  audit  of  this  kind  has  been  conducted  in  the 
state.   Both  audits  were  patterned  after  similar  efforts 
conducted  by  the  State  of  Idaho.   Montana  should  ultimately 
develop  a  unique  process  which  may  better  address  the  needs 
of  its  resource  managers  and  provide  a  better  format  for 
continuing  BMP  assessment  in  the  future. 

METHODOLOGY:  THE  ASSESSMENT  PROCESS 
The  two  fundamental  objectives  in  the  design  of  this  study 
were  to: 

1.  Minimize  opportunity  for  individual  or  sampling 
bias. 

2.  Utilize  as  much  of  the  Environmental  Quality 
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Council's  BMP  study  design  as  possible  to  provide 
for  the  comparison  of  results. 
Areas  of  concern  in  study  design  included: 

1.  Identification  and  delineation  of  drainage  basins 
of  greatest  concern  to  water  quality  problems. 

2.  Avoiding  bias  in  site  selection. 

3 .  Avoiding  bias  in  team  selection  and  team 
composition. 

4 .  Maintianing  consistency  in  site  ratings  between 
teams  evaluating  different  timber  sales. 

5.  Appropriate  distribution  of  sample  sites  across 
ownership  classes. 

6.  Distribution  of  sample  sites  across  erosional 
hazard  classes. 

Sampling  involved  a  total  of  52  timber  sales  or  cutting 
units,  harvested  and  completed  between  1986-1988.   Timber 
sales  or  cutting  units  were  first  compiled  by  ownership 
classes  into  pools  of  candidates  that  met  the  age 
requirement  for  harvest  activity,  and  were  within  2  00  feet 
of  a  stream  channel. 

The  number  of  sales  to  be  visited  in  each  major 
drainage  was  proportional  to  the  total  timber  harvest  volume 
in  those  drainages  between  1986  and  1988,  but  every  major 
drainage  area  in  the  basin  had  at  least  1  timber  sale  audit. 
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STUDY  AREA 

The  Flathead  River  Basin  encompasses  nearly  six  million 
acres  of  land  and  water.   A  number  of  contributing 
watersheds  define  the  boundary  of  the  Basin:  1)  Swan  River 
and  contributing  watersheds;   2)  North,  Middle  and  South 
Forks  of  the  Flathead  River;   3)  Stillwater  River;  4)  Tally 
Lake;  and  5)  Little  Bitterroot  River.   Table  1  provides  a 
summary  of  how  the  timber  sales  were  distributed  across  the 
Flathead  River  Basin. 
Ownership  Classes 

Timber  sales  to  be  audited  were  stratified  into  4 

ownership  classes.   The  number  of  timber  sales  that  were  to 

be  audited  in  each  class  was  roughly  proportional  to  the  to 

the  ratio  harvest  volumes  among  the  ownerships.   There  were 

no  fewer  than  5  sales  audited  for  any  ownership  class. 

Distinctions  among  ownership  classes  were  made  to  see  if 

ownership  was  related  to  BMP  compliance.   The  4  ownership 

classes  were: 

#  of  sales 
Audited 


1.  Flathead  National  Forest   (USFS)       22 

2 .  Department  of  State  Lands  6 

3.  Plum  Creek  Timber  Co.  19 

4.  Non-industrial  private  5_ 

52 


Erosional  Hazard  Classes 

The  objective  of  stratifying  the  timber  sale  audits  by 
erosion  hazard  class  was  to  achieve  equal  representation  for 
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Table  1.   Distribution  of  timber  sales  in  the  Flathead 
Basin. 

#  OF  SALES  AUDITED 


PLUM 

STATE 

NON- 

PRINCIPAL  WATERSHEDS 

USFS 

CREEK 

LANDS 

INDUST 

TOTALS 

SWAN  RIVER 

6 

8 

3 

1 

18 

TALLY  LK  /  LITTLE 

BITTEROOT 

7 

10 

0 

2 

19 

NORTH  FORK  FLATHEAD 

RIVER 

4 

10 

1 

0 

5 

SOUTH  FORK  FLATHEAD 

RIVER 

2 

0 

0 

0 

2 

MIDDLE  FORK  FLATHEAD 

RIVER 

2 

0 

0 

0 

2 

STILLWATER   RIVER 

0 

0 

2 

0 

2 

WHITEFISH   RIVER 

0 

0 

0 

1 

1 

FLATHEAD  LAKE 

1 

1 

0 

1 

3 

TOTALS         22     19       6       5       52 

the  wide  variety  of  landtypes  identified  in  the  Flathead 
Landtype  Inventory.   It  was  realized  that  our  sample  size 
limits  and  the  existing  range  of  site  conditions  would  not 
allow  a  full  and  adeguate  sample  of  all  erosion  hazard 
conditions.   Nevertheless,  this  stratification  provided  for 
a  broad  representation  of  erosion  classes  while  allowing  us 
to  meet  additional  selection  criteria. 

Stratification  of  the  audit  pool  at  this  level  was 
intended  to  see  if  the  level  of  BMP  compliance  was  related 
to  the  erosion  hazards  which  are  experienced  on  a  site. 
Based  on  the  Flathead  Landtype  Inventory,  sale  sites  were 
classified  as  high,  medium  or  low  erosion  hazard.   Those 
classes  were  based  on  average  slope  and  soil  erodibility 
indicated  by  geologic  parent  materials.   The  placement  of 
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sites  in  classes  was  then  field  tested  by  having  team 
members  try  to  assess  the  potential  for  on-site  erosion,  and 
place  a  timber  sale^into  one  of  these  categories. 

SITE  SELECTIONS 
Site  selection  criteria,  as  follow,  were  adopted  from 
the  1988  statewide  assessment  of  BMPs  done  by  the  EQC,  with 
minor  modifications. 

1.  Timber  sales  harvested  between  January  1986  and 
December  1988 

2 .  Site  preparation  completed 

-  slash  piled  and  burned  or  awaiting  burning 

-  slash  broadcast  burned  or  scheduled 

3.  Volume  harvested  >  or  =  7  MBF/Acre 

4.  Cutting  unit  size  of  approximately  5  acres  or 
larger 

5.  Timber  harvesting  occurred  within  a  2  00'  distance 
to  a  stream  channel  (intermittent  or  perennial) 

Database  For  Site  Selection 

Information  needs  and  sources  of  information  for  the 
ownership  classes  are  listed  below. 
Information  Needs: 

1.  Total  volume  of  timber  harvested,  by  ownership,  for 
the  study  area  and  period  in  time. 

2.  Sale  specific  information  for  the  study  period. 

a.  volumes  harvested  by  sale  or  cutting  unit 

b.  method  of  harvest 
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c.  identify  sale  related  road  construction 

d.  method  of  site  preparation 

e.  soils  and/or  parent  material 

f.  status  of  sale,  i.e.,  opened  or  closed 
3.   Sale  maps  with  legal  description  of  section, 

township  and  range. 
Sources  of  Information  by  Ownership: 
U.S.F.S 

Flathead  National  Forest  Supervisors  Office 

Kalispell,  MT 
State  Lands 

Department  of  State  Lands,  Forestry  Division 

Missoula,  MT 
Non-Industrial  Private 

Department  of  State  Lands,  Forestry  Division 

11 B"  Hazard  Reduction  Agreement  Files 
Plum  Creek  Timber  Company 

Dept.  of  State  Lands,  Forestry  Div. 

Master  Hazard  Reduction  Agreement  Files 

Plum  Creek  Timber  Company,  Kalispell  Unit 
&  Clearwater  Unit  -  Swan  Office 
Once  a  complete  list  of  sales  for  the  study  period  and 
Basin  had  been  compiled,  sales  which  did  not  meet  the  site- 
selection  criteria  were  eliminated  from  the  audit  pool. 
Using  available  maps,  sales  were  screened  to  meet  the  2  00- 
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foot-to-stream  channel  criteria,  and  verified  either  by 
phone  contact  with  the  appropriate  managing  office  or  by 
field-checks. 

Those  sales  which  met  all  of  the  criteria  were 
summarized,  by  ownership,  in  a  spreadsheet  format.  The 
respective  Map  Unit  (MU)  or  Landtype  (LT)  for  each  sale  was 
identified  using  the  "Flathead  Country  Land  System 
Inventory".   These  MUs  identify  soils  and  slopes  and  make  it 
possible  to  assign  a  hazard  class  to  each  sale. 
Ground-Truthing  Site  Selections 

A  sub-sample  of  sites  were  visited  in  advance  of  the 
field  teams  to  verify  that  they  met  the  site  selection 
criteria.  About  20  percent  of  all  selected  sales  were 
visited.  Most  of  these  site  visits  were  to  Non-industrial 
Private  sales  as  these  were  the  sites  for  which  information 
regarding  the  actual  harvesting  activity  and  proximity  to  a 
stream  channel  was  least  known. 

TEAM  SELECTION 

Audit  teams  were  selected  in  the  following  manor: 

1.  Names  were  solicited  from  participating  member 
organizations  in  the  Flathead  Basin  Cooperative,  as 
well  as  from  organizations  and  groups  which  have 
expressed  an  interest  in  management  of  the 
resources  in  the  Basin. 

2 .  Audit  team  member  selection  criteria  included  a 
familiarity  with  BMPs  or  interest  in  working  with 
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them  and  knowledge  and  experience  in  one  of  the 
following  disciplines:   forestry,  soils,  hydrology, 
engineering,  and/or  fisheries. 
3.   Names  on  this  list  were  organized  into  5  groups, 

based  on  individual's  field  of  knowledge.   A  random 
selection  of  one  individual  from  each  group,  to 
assemble  each  team,  was  conducted.   This  was 
continued  until  3  teams,  each  with  5  members,  were 
assembled. 
4   Each  team  was  structured  to  include  at  least  one 
professional  who  had  participated  in  the  1988  EQC 
BMP  assessments.   This  provided  each  team  with  an 
individual  with  some  level  of  familiarity  with  a 
project  of  this  type.   The  intent  was  to  build  a 
link  of  rating  consistency  between  the  two 
studies. 
Table  2  displays  a  list  of  the  audit  team  participants 
and  their  respective  disciplines. 

THE  PROCESS 
Rating  Forms 

The  Department  of  State  Lands ■  "DSL  Forest  Evaluation 
Worksheet",  was  the  basis  for  the  form  we  used  for  the  BMP 
assessment  (Appendix  A) .   Some  changes  were  made  from  the 
form  used  in  the  EQC's  1988  study,  but  these  were  only  in 
general  organization  and  additions  to  incorporate  the 
amended  BMPs  resulting  from  that  study. 
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Table  2.   Timber  sale  audit  teams. 

TEAM  1 


GEORGE  WILSON 
FRANK  NETHERTON 
DEAN  SIRUCEK 
DON  ALLEY 
VITO  CILIBERTI 
SCOTT  RUMSEY 


SILVICULTURIST 
ENGINEER 

SOIL  SCIENTIST 
CONSERVATION 
HYDROLOGIST 


FLATHEAD  NAT.  FOREST 
PLUM  CREEK  TIMBER 
FLATHEAD  NAT.  FOREST 
TROUT  UNLIMITED 
BLM 


1 
I 

I 


FISHERIES  BIOL.   FISH,  WILDLIFE  &  PARKS 


ALTERNATES:   DENNY  SIGARS  FOR  FRANK  NETHERTON 


JIM  LEHNER 
JEFF  COLLINS 
TOM  WEAVER 
WALLY  PAGE 
ROBIN  MAGADDINO 


TEAM  2 

SILVICULTURIST 
SOIL  SCIENTIST 
FISHERIES  BIOL. 

HYDROLOGIST 
CONSERVATIONIST 


PLUM  CREEK  TIMBER 
DSL  DIVISION  OF  FOR. 
FISH,  WILDLIFE  &  PARKS 
FLATHEAD  NAT.  FOREST 
AUDUBON  SOCIETY 


ALTERNATES : 


STEVE  ROBBINS  FOR  JIM  LEHNER 
ROD  ASH  FOR  ROBIN  MAGADDINO 
PHYLLIS  SNOW  FOR  WALLY  PAGE 


DEBBIE  MANLEY 
AL  SORENSON 
STEVE  TRALLES 
MIKE  ENK 
BILL  SCHULTZ 
KEITH  ENGEBRETSON 


TEAM  3 

SILVICULTURIST 
ENGINEER 
HYDROLOGIST 
FISHERIES  BIOL. 
HYDROLOGIST 
FORESTER 


FLATHEAD  NAT.  FOREST 
FLATHEAD  NAT.  FOREST 
WATER  QUALITY  BUREAU 
FLATHEAD  NAT.  FOREST 
DSL  DIVISION  OF  FOR. 
MONTANA  LOGGING  ASSOC. 


ALTERNATES:   BILL  BASKO  FOR  DEBBIE  MANLEY 

KIT  SUTHERLAND  FOR  STEVE  TRALLES 


Recognizing  that  these  new  BMPs  were  not  available 
prior  to  1988,  they  were  identified  as  separate  line  items 
on  the  rating  form  and  will  serve  as  baseline  data  for 
future  assessments  of  BMPs. 
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Field  Work 

Prior  to  actually  performing  any  timber  sale  audits, 
the  teams  were  gathered  for  a  one  day  "Training  Audit", 
which  was  conducted  on  May  30,  1989.   The  trial  was  held  in 
the  Tally  Lake  District  of  the  Flathead  National  Forest  and 
provided  an  opportunity  to  introduce  the  process  of  field 
assessment  of  BMPs  and  provide  team  members  some  hands-on 
experience  and  time  to  become  familiar  with  the  rating  forms 
and  evaluation  procedures. 

Procedures  for  evaluating  a  site  were  presented  to  all 
teams  and  teams  had  a  chance  to  conduct  evaluations  on  the 
same  2  timber  sales.   At  each  sale,  teams  were  asked  to 
share  their  results  of  these  evaluations  with  the  entire 
group.   Discrepancies  in  ratings  between  groups  were 
identified  on  each  site  and  discussions  were  held  in  an 
attempt  to  standardize  rating  methods.   The  goal  was  to 
initiate  a  consistent  method  of  addressing  the  practices  on 
a  sale,  and  then  carry  those  methods  through  the  duration  of 
the  field  work. 

For  the  remainder  of  the  regular  field  season  a  field 
coordinator  accompanied  each  team  to  each  timber  sale  and 
worked  to  facilitate  discussions  within  groups,  and  maintain 
rating  consistency. 

Teams  were  scheduled  to  audit  approximately  18-2  0 
sites,  which  required  spending  8  days  in  the  field.   Table  3 
summarizes  the  field  schedule  for  each  team. 
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Table  3  -  The  Flathead  Basin  Timber  Sale  Audit  Schedule 


DATE 


JULY  13 
JULY  14 
JULY  27 

JULY  28 
AUG  2 

AUG  3 
AUG  4 

AUG  17 


AUDIT  # 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 
18. 
19. 
20. 


TEAM 

SALE  NAME 


,29 


GOAT  ROT 

LOWER  CILLY  CREEK 
ELK-COLD  CREEK  5-28 
MISSION  BUTTE  #25 
NAPA  GOAT   #4 
DEAD  ON  IT'S  FEET 
GOAT  CANYON 
UPPER  MCGINNIS  #23 
WOODS  BAY 
DUNN  TEPEE  #4 
STAR  SHEPARD  #22 
(STAR  MEADOWS) 
NINKO  MILLER  #6 
(WHITEFISH  RIVER) 
MOUNT  CRK  OSR  #16 
BROWN  MEADOW  LPP 
(BROWN  MEADOW) 
LOWER  TAMARACK 
G.  BRANCH  II 
WELCOME  PICKLES 


OWNERSHIP 

DRAINAGE 

STATE 

SWAN 

STATE 

SWAN 

USFS 

SWAN 

USFS 

SWAN 

USFS 

SWAN 

PC 

SWAN 

PC 

SWAN 

USFS 

N.FK 

PC 

LAKE 

USFS 

TALLY 

USFS 

TALLY 

NIPF 

TALLY 

USFS 

N.FK 

NIPF 

WH.FISH 

USFS 

TALLY 

PC 

TALLY 

NIPF 

TALLY 

PC 

TALLY 

PC 

TALLY 

PC 

TALLY 

TEAM  2 

DATE 

AUDIT  # 

SALE  NAME 

OWNERSHIP 

DRAINAGE 

JULY 

6 

1. 

EWING  DOG  III  &  EXT. 

STATE 

STILLWTR 

2. 

SWIFT-ANTICE  #7 

STATE 

STILLWTR 

JULY 

7 

3. 

LOWER  TAMARACK 

PC 

TALLY 

4. 

A-B  LODGEPOLE 

PC 

TALLY 

5. 

BERNARD  FLAT 

PC 

TALLY 

JULY 

10 

6. 

BILL  CREEK  #2 

USFS 

TALLY 

7. 

LOGAN  FALLS  #4 

USFS 

TALLY 

8. 

(STAR  MEADOWS) 

NIPF 

TALLY 

JULY 

11 

9. 

STOPHER  GOFOR  #3 

USFS 

SWAN 

10. 

SWEET  MARY  #12A 

USFS 

SWAN 

JULY 

31 

11. 

RIVERSIDE  CANYON  #10 

USFS 

S.FK 

12. 

CIRCUS  PEAK   #14F 

USFS 

S.FK 

AUG 

1 

13. 

NAPA  GOAT  #4 

USFS 

SWAN 

14. 

HUNGRY  BEAR 

PC 

SWAN 

AUG  22 

15. 

STONER  CREEK  #40 

USFS 

LAKE 

16. 

(PABLO-ASHLEY  CRK.) 

NIPF 

LAKE 

AUG  23 

17. 

LION  FLAT 

PC 

SWAN 

18. 

JIM  CORNICE 

PC 

SWAN 
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Table  3 

(continued)       TEAM  3 

JULY  17 

1. 

NORTH  FACE  #4 -4 A 

USFS 

TALLY 

2. 

SANDERS  HAND  #27 

USFS 

TALLY 

3. 

(STAR  MEADOWS) 

NIPF 

TALLY 

JULY  18 

4. 

TERRACE  HILL  #14 

USFS 

MID.FK 

5. 

MIDDLE  FORK  LP  #21 

USFS 

MID.FK 

AUG  8 

6. 

SOUTH  FORK  COAL  CRK 

STATE 

N.FK 

7. 

COAL  RIDGE  #21 

USFS 

N.FK 

8. 

UPPER  COAL  #5 

USFS 

N.FK 

AUG   9 

9. 

NAPA  GOAT  #4 

USFS 

SWAN 

10. 

WEST  JIM 

PC 

SWAN 

AUG  15 

11. 

LOWER  TAMARACK 

PC 

TALLY 

12. 

RED  CLEAR 

PC 

TALLY 

13. 

SOUTH  REDGATE 

PC 

TALLY 

AUG  16 

14. 

HOLLAND  PIERCE  #6 

USFS 

SWAN 

15. 

OWL  ONE  QUARTER 

PC 

SWAN 

SEPT  6 

16. 

SQUAW  CAMP 

PC 

TALLY 

17. 

BROOK  EAST 

PC 

TALLY 

SEPT  7 

18. 

SWAN  RIVER 

STATE 

SWAN 

19. 

SOUTH  COLD 

PC 

SWAN 

20. 

(SALMON  PRAIRIE) 

NIPF 

SWAN 

ABBREVIATIONS  USED: 

USFS   -  U.S.  FOREST  SERVICE 

STATE  -  DEPT.  OF  STATE  LANDS 

PC   -    PLUM  CREEK  TIMBER  COMPANY,  INC. 

NIPF  -   NON-INDUSTRIAL  PRIVATE  FOREST  LAND 

LAKE  -   LAND  ADJACENT  TO  OR  DRAINING  INTO  FLATHEAD  LAKE 

MD.FK  -  MIDDLE  FORK  OF  FLATHEAD  RIVER 

TALLY  -  REGIONAL  DRAINAGE  AREA  WEST  OF  KALI SPELL/ WHITEFISH, 

INCLUDES  ASHLEY  CREEK. 
SWAN  -   DRAINAGE  AREA  ULTIMATELY  REACHING  SWAN  LAKE 
N.FK  -   NORTH  FORK  OF  FLATHEAD  RIVER 
S.FK  -   SOUTH  FORK  OF  FLATHEAD  RIVER 

NOTE:   SALE  NAMES  IN  BOLD  PRINT  ARE  THOSE  THAT  WERE  USED  IN 
THE  RATING  CONSISTENCY  COMPARISON  AMONG  TEAMS. 


Three  timber  sales  were  designated   "comparative" 
sales.   Each  team  rated  these  same  3  sales  throughout  the 
course  of  their  field  work  to  provide  some  data  on  how  teams 
compare  to  one  another  with  respect  to  assigning  ratings  to 
BMP  application  and  effectiveness.   This  information  is 
discussed  in  greater  length  shortly. 
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The  field  routine  consisted  of  meeting  at  some  central 
location  on  the  morning  of  the  field  day.   Teams  generally 
traveled  together,  in  one  vehicle,  to  the  audit  site.   A 
field-coordinator  accompanied  each  team  to  facilitate 
participation  by  all  members,  stimulate  discussion,  and 
record  field  data  on  the  designated  rating  forms.   This 
individual  also  monitored  consistency  in  ratings  from  site 
to  site  and  between  teams. 

On  the  site,  teams  were  provided  maps  and  audit  forms 
and  briefed  by  a  representative  of  the  land  ownership  which 
was  audited.  The  representative  provided  the  teams  with 
background  information  on  the  silvicultural  prescription, 
timing,  and  associated  practices  relating  to  the  entry  we 
evaluated. 

Team  members  were  encouraged  to  stay  relatively  close 
together  while  walking  the  site,  to  the  extent  possible,  to 
stimulate  discussion  of  problems  as  they  were  observed. 
This  also  helped  in  resolving  difficult  ratings  during  the 
completion  of  the  rating  form  later. 

Teams  typically  spent  1-2  hours  surveying  the  condition 
of  a  completed  timber  sale.   Beginning  in  the  riparian 
areas,  teams  worked  their  way  through  these  areas  noting  the 
condition  of  the  "streamside  management  zone"  (SMZ) .   From 
the  SMZs,  teams  worked  toward  portions  of  the  sale  which 
were  potential  contributing  areas  for  sediments  to  stream 
channels.   These  included  any  skid  trails,  fire  lines,  road 
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surfaces  or  tributaries  to  the  main  channel.   All  stream 
channels  and  potential  wet  areas,  including  ephemeral  draws, 
were  inspected  for  evidence  of  washing  or  sediment 
transport.    Although  a  thorough  coverage  of  the  entire  sale 
acreage  was  desired,  it  wasn't  always  obtainable  due  to 
large  areas  and  limited  time.   Teams  concentrated  on  the 
riparian  areas  which  are  recognized  as  most  critical  to  soil 
and  water  protection. 

Inspections  focused  on  the  applicability  of  practices 
used,  based  on  the  "needs"  of  a  site.   The  needs  of  a  site 
were  subjectively  evaluated  by  a  combination  of  the  physical 
site  characteristics  (i.e.,  the  potential  for  erosion),  the 
silvicultural  prescription  and  selected  harvesting  system. 
Other  considerations  included  "delivery  potential"  to  stream 
channels,  and  whether  these  activities  contributed  to 
increased  efficiencies  for  sediment  transport  off -site.   If 
increased  efficiencies  were  observed,  then  it  was  generally 
felt  that  other  practices  should  have  been  used  to  mitigate 
this. 

In  addition,  interpretation  of  the  effectiveness  of  the 
practices  used  in  preventing  water  quality  degradation  was 
rated.   This  was  assessed  by  observing  the  amount  of  surface 
erosion  now  occurring,  the  probability  of  delivery  to  a 
stream  channel  and  the  extent  of  area  over  which  it  was 
occurring.   Most  sales  inspected  were  nearly  2  years  old, 
which  diminished  the  likelihood  of  further  impacts,  and 
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meant  most  erosion,  if  any,  had  already  occurred. 
Therefore,  team  members  had  to  look  for  evidence  of  past 
erosion,  such  as  sediment  detention  behind  woody  debris,  and 
erosion  pavements  from  which  the  fine  soil  fraction  had  been 
removed. 

Upon  completion  of  the  inspection,  and  while  still  on 
the  site,  a  team  gathered  to  discuss  their  observations. 
Each  team  completed  one  rating  form  per  site.   The  field 
coordinator  acted  as  field  recorder  and  discussion 
moderator,  read  through  each  practice  and  asked  for 
suggestions  on  assigning  a  separate  rating  value  for  both 
"application  and  effectiveness".   Upon  obtaining  ratings  on 
the  last  practice,  the  coordinator  summarized  the  general 
overview  of  the  ratings  given,  to  provide  one  last  time  for 
additional  comments.   Each  practice  received  a  subjective 
rating  on  2  variables,  "application"  of  the  practice  and 
it's  "effectiveness",  with  values  assigned  ranging  from  1 
through  5.   The  meanings  of  these  values  can  be  found  on  the 
rating  form  in  Appendix  A  . 

RATING  OF  BNPS 

The  on-site  assignment  of  subjective,  qualitative  BMP 
rating  values  by  an  interdisciplinary  team  is  a  complex 
process.   It   relies  on  successful  interactions  among  a 
number  of  professionals  with  sometimes  conflicting 
objectives,  differing  opinions  and  experiences  working  with 
BMPs.   To  arrive  at  a  consensus  rating  for  this  process 
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requires  that  individuals  pool  their  knowledge  and 
experience  in  natural  resource  management.   Compounding  the 
complexity  of  the  process  is  the  variation  in  site 
conditions  and  silvicultural  treatments  from  site-to-site, 
resulting  in  teams  rarely  viewing  the  same  set  of 
circumstances.   With  no  two  sites  alike  and  vaguely  defined 
conditions  associated  with  the  ratings  scale,  the  process 
provided  an  opportunity  for  a  considerable  amount  of 
individual  interpretation.   Measures  were  taken  to  control 
this  variability  and  quantify  it. 

The  "calibration  audits"  provided  an  opportunity  for 
individuals  and  team  members  to  become  familiarized  with  how 
these  numerical  ratings  should  be  assigned  to  reduce  some  of 
the  subjectivity  and  variability  between  ratings  within 
teams  and  across  teams.   In  addition,  the  exercise  was 
necessary  to  establish  a  process  consistent  with  that  of  the 
EQC  assessments.  Additional  criteria  were  reviewed  at  this 
meeting  to  assist  teams  in  the  assignment  of  the  various 
rating  values. 
Application  of  Practices 

Practice  application  was  rated  with  respect  to  the 
intent  of  each  BMP.   To  establish  the  application  rating, 
several  considerations  are  blended  into  one  rating  number. 

Before  teams  could  assign  a  numerical  rating  for  each 
practice  on  the  audit  form,  they  considered  the  "need"  for 
each  practice  to  mitigate  erosion.   If  a  need  was  not 
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identified,  the  practice  received  a  Not  Applicable  (NA) 
rating.   The  following  questions  helped  to  identify  "need": 

1.  What  was  the  erosion  potential  of  the  site  prior 
to  disturbance,  considering  site  conditions  such 
as  soils,  slope,  topography,  vegetative  cover, 
soil  moisture  and  prior  management  activity? 

2 .  How  did  that  potential  change  with  the  proposed 
management  activities,  i.e.,  road  construction, 
timber  sale  prescription,  landing  and  skid  trail 
location? 

The  actual  assessment  of  the  erosion  mitigation  "needs 
of  a  site  were  based  on  the  following: 

1.  Observation  of  the  magnitude  and  extent  of 
erosion  which  had  or  was  presently  occurring. 

2.  Professional  judgement  of  the  perceived  potential 
for  erosion  to  continue. 

Once  it  was  established  that  erosion  mitigation 
measures  were  needed,  teams  evaluated  the  adequacy  of  the 
practices  which  were  applied.   This  involved  reconsidering 
items  #1  and  #2  above  to  determine  what  the  appropriate 
practices  should  have  been  to  address  these  needs.   Further 
considerations  included: 

3.  Establishing  what  a  preferred  practice  might 
have  been  to  completely  mitigate  any  erosion. 

4.  Assessing  the  economical  and  logistical 
limitations  associated  with  applying  that 
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practice. 
5.   Balancing  items  #3  and  #4  to  identify  a  practice 
that  would  prevent  impacts  to  soil  and  water  and 
still  meet  the  intent  of  the  BMPs. 

The  actual  assignment  of  a  rating  value  involved 
comparing  what  the  teams  identified  as  the  "correct" 
application  of  practices,  from  #5  above,  to  the  actual 
practices  which  were  conducted.   At  times,  this  required 
teams  to  return  to  the  documented  BMPs  to  attempt  to  clarify 
a  BMP's  intent,  either  specified  or  implied. 

The  following  information  summarizes  some  of  the 
special  considerations  associated  with  each  of  the  5  rating 
values. 
5  -  Operation  Exceeds  Requirements  of  BMPs 

This  rating  was  assigned  to  practices  which 
demonstrated  additional  care  or  special  treatment  to 
mitigate  erosion,  beyond  that  which  is  specified  or  implied 
in  the  BMP.   This  may  be  reflected  in  the  areal  extent  and 
frequency  that  a  practice  was  applied,  or  in  higher 
standards  of  construction  for  the  erosional  preventative 
features . 

A  practice  was  considered  in  relation  to  the 
accomplishment  of  silvicultural  objectives.   For  example,  a 
5  was  not  issued  in  cases  where  an  extremely  large 
streamside  management  zone  was  left  intact  for  wildlife 
considerations . 
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4  -  Operation  Meets  BMP  Requirements 

The  practice  met  the  intent  of  the  BMP  and  the 
frequency  and  standards  of  application  were  deemed 
sufficient  to  mitigate  erosion.   A  very  limited  amount  of 
erosion  was  observed  and  soils  were  determined  to  be 
stabilized.   A  team  would  generally  not  feel  a  need  to 
offer  recommendations  for  alternative  practices. 
3  -  Minor  Departure  From  Intent  of  BMP 

Generally,  this  rating  was  used  in  cases  where  an 
alternative  practice  should  have  been  selected  and  practices 
should  have  been  applied  with  a  higher  standard  and/or 
frequency. 

Erosion  was  exceeding  that  which  is  acceptable  under 
the  intent  of  the  BMP  and/or  the  risk  of  potential  erosion 
was  judged  to  be  high  enough  to  justify  improved  practice 
application. 

The  term  "minor"  can  be  interpreted  to  mean  a  departure 
of  relatively  small  magnitude  distributed  over  either  a 
localized  area  or  broadly  over  an  entire  sale  area  and 
where  a  potential  for  impact  has  been  identified  as  being 
low.   A  "Minor"  may  also  be  interpreted  to  be  a  larger 
magnitude  of  departure  applied  over  a  very  limited  area. 

"Minor"  ratings  also  commonly  applied  to  circumstances 
where  the  level  of  required  protection  for  the  resource  was 
sufficiently  low.  It  is  possible  that  a  practice  may  not 
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have  been  applied  at  all,  and  yet  it  received  only  a  "minor 
departure"  because  the  needs  were  judged  to  be  low. 
2  -  Major  Departure  From  Intent  of  BMP 

"Major"  departures  were  those  practices  with  departures 
which  were  large  in  magnitude  and  with  an  increased 
frequency  of  misapplication,  and  displayed  a  greater 
potential  for  damaging  the  soil  and  water  resource. 

Extensive  erosion  was  generally  observed  either  in  or 
immediately  adjacent  to  stream  channels,  or  sediment  was 
observed  moving  on-site  in  such  a  way  that  the  probability 
of  sediment  reaching  the  streams  was  high. 
1  -  Gross  Negligence 

This  rating  applied  to  those  instances  where  the  risks 
to  soil  and  water  resources  were  most  obvious  and  yet  no 
evidence  existed  to  indicate  that  practices  were  altered 
to  provide  for  resource  protection.   Ratings  were  influenced 
by  the  actual  departures  observed,  their  magnitude,  the 
areal  extent  of  the  practices,  and  the  number  of 
opportunities  which  existed,  on  a  given  sale,  to  apply  a 
specific  practice. 

In  general,  the  number  of  opportunities  to  apply  a 
particular  practice  were  a  function  of  the  amount  and 
location  of  road  construction,  the  acreage  of  the  harvest, 
and  the  proximity  of  harvesting  or  equipment  operation 
adjacent  to  stream  channels  or  wet  areas. 
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Stream  crossings  are  an  example  of  this  variable.   A 
sale  that  has  only  one  designated  stream  crossing  associated 
with  it,  and  that  one  location  has  a  poorly  constructed 
crossing  could  be  rated  as  a  "major"  departure.   This  is 
because  the  sale  provided  limited  opportunities  to  apply  the 
practice,  and  in  that  one  spot  where  it  was  applied,  it  was 
not  adeguate . 

In  some  situations  the  only  difference  between  a 
"minor"  and  "major"  departure  was  the  location  of  each 
practice  within  the  given  sale  area,  and  how  that  location 
influenced  the  potential  for  impacts  to  soil  and  water 
resources.   Teams  found  the  assignment  of  a  single  rating 
value  to  represent  the  level  of  BMP  application  across  the 
entire  sale  area  to  be  difficult  at  times.   This  was 
particularly  noticeable  in  situations  where  the  physical 
site  characteristics  varied  considerably  across  a  site. 
Effectiveness  of  Practices 

This  rating  scale  asks  the  guestion:  "Has  the 
application  or  misapplication  of  a  particular  forest 
practice  increased  the  likelihood  of,  or  actual  occurrence 
of,  surface  sediment  in  the  stream  channels?" 

The  site  selection  criteria  requirement  -  that  all 
sales  sampled  have  harvesting  activity  within  approximately 
200  feet  of  a  stream  channel  -  was  intended  to  provide  a 
look  at  only  those  sales  with  a  higher  likelihood  of  impact 
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to  water  quality.   However,  we  were  not  assessing  direct 
impacts  to  water  quality. 

Impacts  were "defined  as  increased  soil  erosion  because 
of  disturbance  to  soil  surfaces  and  the  development  of  more 
efficient  systems  of  sediment  conveyance  which  contribute  to 
or  increase  the  likelihood  of  sediment  reaching  the  stream 
channels.   Impacts  were  not  linked  in  any  way  to  Montana's 
water  quality  standards,  protection  of  any  beneficial  use, 
or  assessed  based  on  any  physical  measurements  of  the  stream 
channel  itself,  such  as  measurements  of  suspended  sediments 
or  embeddedness .   This  type  of  quantitative  information 
would  be  difficult  to  link  directly  to  logging  practices. 

Teams  observed  several  sites  on  which  livestock  grazing 
impacts  were  clearly  contributing  a  proportionately  greater 
amount  of  sediment  due  to  stream  bank  damage,  than  any 
harvesting  activity  on-site.   However,  this  did  not  prevent 
teams  from  rating  forest  practice  effectiveness  on  these 
sites.   Team  members  felt  that  they  could  adequately 
separate  the  impacts  associated  with  each  of  these  land 
management  activities  at  the  level  which  this  methodology  is 
designed  to  address. 

We  made  the  assumption  that  prevention  of  material 
eroding  down-slope  was  generally  adequate  protection  of  the 
soil  resource,  but  the  decision  on  what  rating  to  assign  the 
practice  contributing  to  this  departure  was  dependent  on 
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factors  which  would  influence  it's  ability  to  reach  a  stream 
channel  and  ultimately  leave  the  site. 

Some  of  the  factors  considered  included: 

1.  The  network  of  existing  channels  or  surfaces  of 
conveyance  on-site,  i.e.,  roads  or  skid  trails. 

2.  The  proximity  of  the  disturbed  soils  to  these 
channels  or  surfaces  of  conveyance. 

3 .  The  present  condition  of  the  vegetative  cover  and 
the  perceived  ability  of  the  site  to  stabilize. 

4.  The  area  of  soil  disturbance. 

5.  Whether  the  soil  was  determined  to  be  in  transit 
downslope  to  a  channel. 

6.  Topography  and  slopes  providing  energy  for 
transport. 

7.  Size  and  effectiveness  of  vegetative  buffer  below 
the  disturbance. 

8.  Evidence  of  soil  in  stream  channels,  linked  to 
forest  practices. 

9.  Evidence  of  rilling,  gullying,  or  the  presence 
of  an  erosional  pavement  indicating  fine  textured 
soil  particles  had  been  transported. 

10.   Evidence  of  sediment  detention  trapped  by  woody 
debris,  either  in  channels  and  riparian  areas  or 
in  upland  locations. 
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The  consideration  and  weighting  of  these  factors  in 
team  discussions  determined  the  rating  which  a  practice 
received  with  respect  to  impacts.      * 

The  following  summary,  by  rating  value,  presents  a 
general  description  of  the  conditions  which  warranted  a 
given  rating.  These  were  determined  from  a  review  of  the 
comments  on  the  audit  forms. 

5  -  Improved  protection  of  resources  over  pre-project 
conditions 

This  rating  was  used  in  situations  where  either  natural 
erosion  or  erosion  from  a  previous  entry  was  mitigated  by 
practices  conducted  on  the  audited  sale. 
4  -  Adequate  protection  of  soil  and  water  resources 

There  was  little  evidence  of  erosion  taking  place  and 
low  probability  that  any  would  occur  in  the  future. 
Probability  of  any  sediment  reaching  the  stream  channel  from 
practices  was  determined  to  be  low. 
3  -  Minor  and/or  temporary  impacts  on  soil/water  resources. 

Erosion  occurring  on-siteas  a  result  of  the  forest 
practices.   Sediment  has  been  transported  on-site  and  the 
majority  is  presently  being  trapped  there  by  terrain,  debris 
or  vegetation.   The  probability  of  continued  erosion  is  low. 
Soil  disturbances  may  be  excessive  in  locations  and  may  be 
reaching  channel.   The  amount  of  sediment  contribution  to 
streams  is  not  viewed  as  significant  in  the  context  of  this 
sale. 
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This  rating  was  also  given  when  areas  of  soil 
disturbance  showed  minimal  evidence  of  transport  on-site, 
yet  the  disturbance  was  positioned  in  such  a  location  that 
it  was  a  potential  threat  to  the  stream  channel  under  the 
force  of  gravity  alone. 

Low  gradient  stream  channels  and  their  limited  ability 
to  move  sediment  off -site  was  also  a  factor  in  this  rating, 
as  was  the  ability  of  a  site  to  stabilize  the  sediments. 
Improved  practice  application  would  have  prevented  these 
impacts. 
2  -  Major  detrimental  impacts/  primarily  short-term 

Significant  amounts  of  erosion  were  occurring  on-site 
either  immediately  adjacent  to  a  stream  course  or  on 
contributing  surfaces,  such  as  a  roads.   Direct  contribution 
of  sediment  into  the  stream  channel  was  observed  at 
unsatisfactory  levels,  or  the  risk  of  delivery  was  so  great 
that  it  was  judged  only  a  matter  of  time  before  it  was 
realized. 

Soils  would  not  be  stabilized  by  the  following  season 
of  activity.    Streambank  and  channel  damage  resulting  from 
harvest  activity  was  observed  on-site. 

Short-term  recovery  was  interpreted  to  mean  site 
stabilization  within  2-4  years  following  completion  of  the 
operation.   Corrective  measures  were  needed  to  mitigate  the 
erosion. 
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1  -  Major  detrimental  impacts,  damage  extensive,  recovery 
expected  to  be  slow. 

Similar  to  #2,  but  the  difference  was  the  estimated 
time  required  for  adequate  stabilization  of  soil. 

This  rating  process  rarely  resulted  in  unanimous 
agreement  on  rating  values,  but  a  majority  of  each  team 
frequently  agreed  on  the  scores  which  were  assigned.   In  a 
few  cases,  team  members  were  split  or  indecisive  and  a  tie 
breaking  vote  was  called  to  arrive  at  a  rating. 

Certain  site  characteristics  were  noted  on  sales  in 
which  a  consensus  was  difficult  to  reach.   Older  sales 
commonly  didn't  give  clear  indications  of  the  impacts  which 
may  actually  have  occurred  immediately  following 
disturbance.   Healing  of  the  soil  surface  and  revegetation 
on  these  sites  made  assessments  of  surface  erosion 
difficult.   Stream  channels  which  were  poorly  defined  also 
produced  more  disagreement  over  the  need  for  protection  and 
resulted  in  a  greater  diversity  of  opinion  in  the  ratings. 

For  the  most  part,  if  practices  were  adequately 
applied,  the  effectiveness  in  protecting  the  resource  was 
also  adequate.  This  close  correlation  is  reflected  in  the 
RESULTS  section  (Table  5) .  On  several  sites,  practices 
received  differing  ratings  for  application  and 
effectiveness,  which  did  not  support  the  direct  link  between 
these  two  variables.   Concerns  were  expressed  over  what 
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these  cases  might  imply.   Two  of  these  situations  were 
observed: 

1.  An  adequately  rated  practice  application  which 
results  in  an  inadequate  protection  of  resources 
may  point  out  at  least  two  concerns: 

a)  The  BMP  language  has  not  been  specific  enough 
in  addressing  site  conditions  which  are  more 
erodible  than  others.   What  may  be  considered 
adequate  on  one  site,  for  practice 
application,  may  be  inadequate  for  another 
more  sensitive  site.   Teams  viewed  sites  on 
highly  erodible  soils  that  reflected  obvious 
care  in  planning  and  execution,  and  yet  the 
results  were  impacts  to  soil  and  water. 

b)  It  may  illustrate  the  trade-off  which  is  made 
between  managing  lands  for  resource 
production  and  the  minimizing  of  non-timber 
resource  impacts . 

2.  Practices  which  were  needed  and  inadequately 
applied,  and  yet  no  impacts  were  observed.   This 
could  indicate: 

a)  Incorrect  interpretation  of  BMP  language, 

for  which  the  owner  was  fortunate  not  to  have 
experienced  a  combination  of  circumstances 
that  would  produce  an  observable  impact. 

b)  Site  erosion  potential  existed  but  was  not 
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judged  sufficiently  high  to  call  an  impact. 

This  BMP  rating  process  is  highly  subjective,  and  could 
be  significantly  improved  with  work  on  developing  rating 
scales  which  provide  general  guidelines  for  field 
application  by  assessment  teams.   There  is  a  need  to  provide 
some  standardization  to  this  process  without  ignoring  the 
input  of  professional  judgement. 

There  may  be  other  alternative  rating  scales  or 
descriptive  terms  which  may  be  less  open  to  a  wide  array  of 
interpretation.   In  addition,  since  BMP  assessments  will  be 
conducted  in  the  future  utilizing  different  personnel,  it 
might  be  advantageous  to  narrow  the  range  of  interpretation 
so  that  one  could  be  more  assured  that  a  rating  of  a  "3" 
from  the  FBC  study  of  1989  would  carry  the  same  meaning  as  a 
"3"  in  some  future  study. 

DISCUSSION 

Much  of  the  information  gathered  during  this  study  is 
gualitative.   Nonetheless,  it  can  be  valuable  in 
understanding  not  only  how  the  process  was  conducted,  but 
for  guidance  in  future  studies  and  interpretation  of 
results. 

This  section  presents  a  summary  of  observations  and 
suggestions  made  in  the  field  by  audit  team  members,  and 
others,  including  myself,  in  regards  to  all  aspects  of  the 
study.   The  section  is  divided  into  7  sub-sections 
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I- 


addressing  different  components  of  the  BMP  assessment 

process. 

1.   Atmosphere  of  Participation 

Studies  which  seek  to  examine  the  possible  inadequacies 
of  resource  management  practices  and  identify  areas  for 
improvements  are  often  viewed  as  threatening  to  individuals 
and  organizations  in  a  number  of  ways. 

Agencies  and  organizations  may  view  them  as  costly  in 
terms  of  a  commitment  of  time  and  may  view  any  findings 
which  indicate  a  less  than  adequate  job  as  possible  "fuel" 
to  ignite  a  movement  toward  forest  regulation.   Individuals 
whose  work  is  being  reviewed  during  the  process  may  harbour 
fears,  real  or  perceived,  of  potential  threat  of 
disciplinary  action  from  management,  or  may  even  relate 
study  outcomes  to  long  term  job  security.   Many  of  the 
individual  sale  administrators  seemed  to  have  only  a  vague 
idea  of  the  intent  of  the  study.   Even  though  explanations 
were  provided,  some  appeared  to  be  uncertain  of  their  role 
and  purpose  and  possibly  more  uncertain  of  the  support  which 
they  had  from  their  upper-management.  Defensive  postures 
during  discussions,  and  a  reluctance  to  admit  that  a  choice 
of  practices  might  not  have  been  the  best  approach,  acted  as 
barriers  to  obtaining  answers  or  moving  forward  with 
discussions. 

The  situation  was  difficult  for  many  of  the 
administrators.   Although  they  were  not  directly  involved  in 
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assigning  rating  values,  the  audit  team  members 
occassionally  had  to  rely  on  administrators'  input  to  weigh 
decisions  on  a  rating.   Questions  regarding  the  practicality 
of  certain  practices  on  site  or  the  timing  of  practice 
application  were  freguently  directed  to  the  administrator. 
Their  response  could  influence  ratings. 

Nobody  likes  to  receive  poor  performance  grades,  for 
they  typically  result  in  reduced  monetary  compensation  or 
opportunities  for  advancement.   For  a  sale  administrator, 
bad  ratings  could  be  viewed  as  threatening  if  management 
doesn't  send  a  clear  message  to  the  contrary.   For  those 
administrators  who  openly  and  honestly,  shared  information 
and  admitted  that  unforseen  problems  had  occurred,  and  that 
practices  could  have  been  improved,  your  honesty  and 
professionalism  was  truly  appreciated  by  the  team  members 
and  myself. 

If  managers  in  this  cooperative  effort  want  a  complete 
and  accurate  report  of  the  level  and  effectiveness  of  BMP 
implementation,  then  there  should  be  a  formal  communication 
from  management  to  the  support  personnel  explaining  the 
project's  objectives  and  soliciting  full  support  from  the 
necessary  staff  personnel.   This  was  lacking  on  this  study 
and  in  some  areas  may  have  been  beneficial  in  improving  the 
cooperation  and  any  educational  benefits  with  respect  to  BMP 
application  and  effectiveness  which  may  have  been  derived. 
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2.   Database  and  Site  Selection 

The  objective  to  provide  a  comparison  of  the  audit 
results  in  the  Flathead  Basin  to  those  of  the  EQCs  1988 
study  restricted  our  choice  of  timber  sales  to  those  which 
met  the  previous  study's  site  selection  criteria. 

Timber  sale  criteria  and  the  associated  information 
needed  for  the  study  were  submitted  to  each  ownership  group, 
except  the  Non-Industrial  Private  whose  data  were  provided 
by  the  Department  of  State  Lands.   All  management  activity 
information  was  returned  to  us  in  different  formats,  and  we 
screened  the  information  for  necessary  data. 

For  each  ownership,  a  "pool"  of  timber  sales  was 
assembled  to  serve  as  a  base  for  site  selection.   The 
following  factors  may  have  potentially  influenced  the  site 
selection  process  by  reducing  the  number  of  timber  sales 
available  for  audit.   In  turn,  these  factors  may  have 
influenced  our  ability  to  accurately  sample  the  population 
of  forest  practices.   These  items  include: 

1.  The  cooperation  of  the  participating 
organizations  in  sharing  their  records. 

2.  The  format,  accuracy  and  completeness  of  the 
database  available  or  provided  by  each  ownership. 

3 .  The  number  of  timber  sales  in  each  ownership 
which  met  all  of  the  study  criteria. 

4.  The  willingness  of  the  non-industrial  ownership 
to  participate  in  the  study. 
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Of  the  items  listed  above,  #2,  #3  and  #4  directly 
influenced  the  site  selection  process  by  limiting  the  total 
available  sites  from  which  to  select,  limiting  the  use  of 
the  random  sampling  process,  and  introducing  bias  into  site 
inclusion  for  the  non-industrial  private  ownership  group. 

For  each  of  the  4  ownership  classes  there  were  unique 
challenges  in  compiling  a  list  of  qualified  timber  sales  to 
sample  from.   The  following  sub-sections  discuss  those 
challenges. 

A.   USFS  Flathead  National  Forest 

A  list  of  timber  sales  on  the  Flathead  National  Forest 
was  provided  by  the  Forest  Supervisor's  office  and  included 
all  timber  sales  which  were  "closed"  between  1986  and  the 
fall  of  '1988.   In  addition,  2  maps  of  each  sale  were 
provided  -  sale  area  and  slash  plan.   The  list  provided  60 
sales  with  contract  number,   contractor,  sale  name,  and  map 
number . 

Some  requested  information  was  not  provided  due  to 
limitations  in  the  Flathead  Forest  database.   Information 
not  provided  included  the  timing  of  harvest  activity  by 
cutting  unit,  acreage  of  cutting  units,  volume  of  timber 
harvested  by  cutting  unit,  silvicultural  treatment 
information  and  road  construction  information.   The  maps  did 
provide  the  general  information  for  determining  the 
"distance  to  stream  channel"  criteria,  and  identified  the 
planned  treatment  and  yarding  method,  new  road  construction, 
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and  in  a  few  cases  provided  summary  tables  for  acreages  of 
cutting  units,  while  others  were  estimated. 

This  format  involved  a  considerable  amount  of  time  in 
handling  and  summarizing  data,  which  were  later  found  to  be 
incomplete.   Some  flawed  assumptions  were  made  at  the 
Supervisor's  office  concerning  the  "age"  of  the  harvest 
activity  for  the  list  of  60  "closed"  timber  sales.   Many  of 
the  60  sales  were  found  to  be  too  old  to  meet  the  study 
criteria  and  had  to  be  discarded  from  our  pool. 

Several  cutting  units  met  the  age  reguirement  but  the 
road  construction  and  sale  layout  could  have  been  as  much  as 
5  to  10  years  old.   This  raised  the  question  of  whether  our 
focus  should  be  on  more  current  roads  and  sale  layout 
practices. 

To  continue  the  verification  process,  those  sites  which 
did  meet  the  initial  criteria  of  "age"  had  to  also  meet  the 
distance-to-stream-channel  requirement  of  2  00  feet.   Sale 
maps  were  scaled  to  measure  the  distance  to  an  identified 
stream  course.   During  the  verification  process  with  the 
District  people  and  on  the  ground,  it  was  learned  that  many 
cutting  units  which  appeared  to  meet  the  requirement  "on 
paper"  were,  in  fact,  more  than  200  feet  away  from  a  stream 
channel  in  layout.   This  required  that  an  alternate  site  be 
selected. 
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Between  these  two  criteria  -  "age  of  harvest"  and  "  2  00 
foot  proximity  to  a  stream  channel",  many  of  the  60  timber 
sales  were  disqualified  from  our  sampling  process. 

"Open"  timber  sale  contracts,  with  completed  slash 
disposal  on  cutting  units  were  proposed  as  possible 
substitutes  to  fill  the  voids  of  the  lost  sales.   Time  would 
not  permit  a  complete  reconstruction  of  the  site  pool  to 
include  all  "open"  sales  with  available  cutting  units. 
District  people  cooperated  by  providing  a  selection  of 
these,  known  to  have  completed  work,  from  which  replacement 
cutting  units  were  randomly  selected. 

For  future  studies  of  this  kind  it  is  suggested  that 
the  Flathead  Forest  database  for  site  selection  be  refined 
to  provide  a  more  current  listing  of  completed  cutting 
units.   The  new  listing  would  contain  cutting  units  from 
both  closed  contracts  and  open  contracts  which  have  had 
their  slash  disposal  completed.   Comments  were  made  by 
personnel  at  the  District  level,  that  information  regarding 
the  stage  of  completion  of  cutting  units  on  open  sale 
contracts  is  available  through  each  District  office.   This 
information  is  current  to  within  6  months  or  less  of  any 
given  point  in  time.   This  would  enhance  our  pool  of  sites 
to  select  from  and  would  serve  both  to  avoid  selecting  sites 
which  are  too  old  to  accurately  assess  practices,  and  may 
also  provide  more  of  the  Forest's  latest  work  with  BMP 
implementation.   In  addition,  much  of  the  non-spatial 
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information  which  was  summarized  from  maps  might  be 
available  in  report  form,  serving  to  reduce  the  costs 
associated  with  the  data  organization  related  to  the  site 
selection  process. 

B.   Non-Industrial  Private 

The  Non-Industrial  Private  ownership  category  had  an 
inadeguate  database  to  identify  sales  meeting  the  study 
criteria.   Many  days  were  reguired  to  build  a  database  and 
verify  the  information.  It  became  cost-prohibitive  to  build 
a  complete  pool  of  candidates  to  choose  from.   Instead,  an 
arbitrary  target  of  approximately  50  timber  sales  was 
identified  as  a  reasonable  number  from  which  to  randomly 
select  5  sites.   These  50  sales  were  identified  using  a 
computerized  summary  of  completed  slash  hazard  reduction  "B" 
agreements,  from  the  Department  of  State  Lands  (DSL) . 
However,  these  50  sites  had  not  yet  been  confirmed  to  meet 
the  2  00  foot  distance-to-stream-  channel  criteria,  nor  had 
the  land  owners  been  contacted  for  their  permission  for 
access  to  their  property.  These  two  factors  which  would 
contribute  to  further  difficulties  in  selecting  an  adeguate 
non-biased  sample  from  this  ownership. 

The  files  of  the  signed  slash-reduction  agreements 
proved  inadeguate  to  provide  information  regarding 
"distance-to-stream-  channel"  criteria.   As  part  of  the 
signed  "Slash"  agreement,  the  landowner  or  contractor  is 
asked  to  provide  a  map  of  the  area  to  be  harvested,  with  an 
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adequate  legal  description.   Records  indicated  that  maps 
were  rarely  provided  by  landowner  or  contractor,  and  the 
State  Lands  slash  administrator  did  not  always  insist  on 
one.   More  recent  slash  reduction  agreement  forms  now  carry 
a  data  field  to  address  water  on  site,  but  are  not  set  up  to 
identify  stream  channels.   The  recent  House  Bill  678,  which 
addresses  the  administrative  responsibility  for  the  BMP 
assessments,  and  the  subsequent  program  to  be  developed  and 
administered  by  the  DSL  may  provide  procedures  for 
documenting  stream  channels  on  non-industrial  private  lands 
and  encourage  a  more  complete  database  for  future  site 
selections. 

Without  accurate  maps  to  check  the  "distance...",  it 
was  extremely  difficult  to  qualify  or  disqualify  these  50 
sales  without  field  checking  or  contacting  each  contractor 
or  slash  administrator  for  confirmation.   Numerous  phone 
calls  were  made  to  loggers  and  slash  hazard  administrators 
to  draw  upon  their  memory  as  to  whether  harvesting  occurred 
adjacent  to  a  stream.   This  proved  equally  inefficient  as 
slash  hazard  administrators  had  difficulty,  rightfully  so, 
remembering  what  had  occurred  on  a  sale  which  was  2  years 
old.   In  addition,  contractors  may  have  several  slash 
agreements  open  at  any  one  time,  making  it  difficult  for  an 
administrator  to  keep  them  straight  in  their  own  minds  with 
respect  to  this  requirement. 
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As  a  sale  was  determined  to  meet  the  eligibility 
criteria,  attempts  were  made  to  locate  landowners  to  obtain 
permission  to  access  their  property.   Typically  such 
attempts  were  met  with  reluctance  and  suspicion  on  the  part 
of  the  landowners.   Of  the  first  14  property  owners 
contacted,  12  declined  to  participate  for  various  reasons. 
This  reluctance  to  participate  reduced  the  pool  of  sites 
considerably.   The  first  2  sites  for  which  permission  was 
received,  were  landowners  who  were  clearly  proud  to  display 
their  planning  efforts,  and  "show-off"  their  work.   It  was 
becoming  clear  that  as  owners  exercised  their  option  of 
non-participation,  any  thoughts  of  this  sample  being 
non-biased  were  quickly  dismissed. 

As  the  inclination  to  eliminate  this  ownership  from  the 
study  grew,  sale  reviews  through  DSL  files  and  phonework 
continued  until  we  had  obtained  5  sales  that  met  study 
criteria  and  for  which  permission  was  obtained.   It  should 
be  noted  that  these  represented  the  first  5  timber  sales 
which  met  all  of  the  study  requirements  and  for  which 
permission  was  granted.   Random  selection  was  not  conducted 
for  lack  of  an  adequate  sized  pool  to  select  from. 

Other  states  which  utilize  BMP  assessments,  such  as 
Idaho,  have  provisions  in  their  forest  practices  legislation 
which  provide  agencies  access  to  non-industrial  private 
lands  for  monitoring  of  BMP  compliance.   Any  future  audits 
in  this  state  should  recognize  the  difficulties  faced  in 
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gaining  participation  from  this  sector,  and  the  influence  it 
has  on  achieving  a  representative  sample  of  practices. 

C.   Plum  Creek  Timber  Company 

The  number  of  timber  sales  from  this  ownership  was 
influenced  strongly,  particularly  in  the  Swan  River 
drainage,  by  a  backlog  of  scheduled  slash  treatment, 
preventing  those  sales  from  being  included  in  our  sample. 
At  the  time  this  database  was  compiled,  over  30  %  of  all 
sales  harvested  between  1986  and  the  spring  of  1989  were 
awaiting  their  slash  treatment.   A  backlog  of  sales 
requiring  their  slash  to  be  treated  had  resulted  from  two 
successive  seasons  of  dry  weather,  which  limited  the  window 
for  slash  burning. 

This  reduced  the  population  of  sales  to  such  a  low 
level  that  virtually  all  of  the  sales  which  met  the  criteria 
in  the  Swan  River  drainage  were  used,  without  a  random 
selection  process.   In  fact,  so  few  were  available  that  in 
order  to  achieve  our  target  sample  number  from  this 
ownership  and  drainage  the  slash  treatment  criteria  was 
relaxed.  This  provided  additional  sales  which  had  all 
harvesting  completed  and  yet  were  awaiting  a  slash  treatment 
prescription.   Only  sales  which  had  slash  prescriptions 
requiring  no  additional  equipment  operation  on  site,  i.e., 
scheduled  for  broadcast  burning,  were  included.   This  was 
necessary  in  order  to  provide  a  target  of  approximately  10 
timber  sales  from  this  ownership  and  drainage  in  our  sample. 
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Future  BMP  assessments  which  rely  upon  unbiased, 
representative  sampling  will  depend  on  the  assumptions  made 
about  the  accuracy  of  the  database  information,  the 
percentage  of  sales  meeting  the  study  criteria  to  the  total 
sales  harvested  during  the  study  period,  and  the  willingness 
of  all  ownerships  to  participate. 

D.   Montana  Department  of  State  Lands 

The  "open"  and  "closed"  timber  sale  contract  files  were 
used  at  the  Forestry  Division  offices  for  compilation  of  a 
list  of  sales  meeting  the  study  criteria.   The  only  problem 
encountered  in  assembling  a  pool  of  sites  was  the  lack  of 
harvest  activity  occurring  within  a  200  foot  distance  to  the 
stream  channel. 

On  several  sales,  cutting  units  were  identified  in  the 
original  "contract  for  bid"  as  meeting  the  proximity  to 
stream  channel  reguirement.   However,  the  files  indicated 
that  upon  completion  of  the  environmental  review  by  the  soil 
scientist,  hydrologist,  and  wildlife  biologist,  cutting 
units  were  either  dropped  from  the  sale  or  sale  boundaries 
were  adjusted.   This  resulted  in  pushing  the  activities 
beyond  the  200  foot  distance  criteria.   Sales  which  were 
conducted  within  this  distance  were  freguently  "salvage" 
operations  for  blowdown. 

Other  units  located  within  200  feet  of  a  stream  channel 
were  usually  too  small  to  gualify  in  the  minimum  acreage  and 
volume  criteria.   Few  new  roads  were  constructed  as  timber 
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on  these  parcels  was  yarded  by  cable  to  an  pre-existing  road 
resulting  in  only  minimal  disturbance  of  the  land. 
3.   Team  Selection 

One  of  the  criticisms  expressed  during  the  formation  of 
the  Cooperative  was  that  the  participants  involved  could  not 
"objectively"  answer  the  question  they  set  out  to  address. 
The  Cooperative  was  likened  to  the  analogy  of  the  "fox 
watching  over  the  hen  house"  and  suggestions  were  made  to 
bring  in  organizations  outside  of  the  immediate  land 
management  ownerships  to  remedy  this. 

Audit  team  selection  is  open  to  the  same  criticisms. 
Originally  it  was  proposed  that  teams  be  constructed  in  such 
a  way  that  no  individual  rated  his/her  own  work  or  rated 
work  which  was  directly  under  their  supervision. 
Understandably,  this  was  intended  to  prevent  bias  or 
prejudice  from  entering  into  the  rating  values. 
Logistically  this  may  have  required  that  some  individual  not 
be  allowed  to  rate  particular  drainages  where  the 
concentration  of  their  previous  work  was  located.   Although 
this  could  have  been  arranged,  the  Cooperative  expressed 
little  concern  over  the  bias  which  might  enter  the  study  as 
a  result  of  this  and  expressed  a  greater  concern  in  getting 
teams  to  audit  practices  across  several  drainages,  instead 
of  concentrating  any  one  team  in  one  drainage. 

Ultimately,  team  members  found  themselves  in  the 
difficult  position  of  rating  their  own  agency  or  company's 
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work.  Some  team  members  were  more  comfortable  with  this  task 
than  others.   However,  on  a  few  sales  where  a  rating 
consensus  was  difficult  to  get,  some  team  members  openly 
admitted  that  their  rating  would  be  biased  and  even  verbally 
expressed  a  desire  not  to  participate  in  the  rating  of  a 
particular  practice.   This  seemed  to  indicate  that  even  some 
audit  team  members  were  having  difficulties  with  assigning 
rating  values  in  an  objective  manner. 

This  occurred  on  only  a  few  sites,  but  points  out  the 
trouble  some  individuals  have  in  openly  critiquing  projects 
they  have  been  associated  with.  Furthermore,  timber  sales 
where  which  a  consensus  was  reached  with  the  least 
discussion,  were  those  in  which  no  audit  team  member  had  a 
personal  interest  in  the  outcome  of  the  ratings,  i.e.,  the 
Non-Industrial  Private  lands. 

The  credibility  of  the  audits  can  be  enhanced  by  taking 
every  practical  step  to  avoid  bias  and  to  provide  accurate 
assessments  from  which  to  build  and  improve  on.   If  this 
means  assigning  team  members  to  rate  only  practices  on 
ownerships  other  than  their  own,  then  it  is  suggested  those 
steps  be  taken. 
4.   BMP  Field  Form 

This  section  is  subdivided  into  two  sub-sections  to 
address   1)  the  ability  to  adequately  rate  practices  on  the 
audit  form,  and  2)  the  rating  scale  and  values  used  on  the 
audit  form. 
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A.   Practices  on  the  Form: 

During  the  field  work,  audit  teams  found  practices 
identified  on  the  field  forms  that  did  not  lend  themselves 
to  accurate,  meaningful  on-site  assessment.   Although 
ratings  could  be  assigned,  it  was  felt  that  the  value 
assigned  might  not  accurately  reflect  the  practice 
application  or  effectiveness. 

This  situation  was  frequently  seen  with  practices  that 
were  either  administrative  in  nature,  such  as  obtaining 
stream  crossing  permits,  or  related  to  the  timing  of  the 
operation.   Both  of  these  situations  involved  an  inability 
to  observe  the  practice  on-site  or  make  a  direct  connection 
between  practice  and  impact.   On  many  of  the  sales,  2  years 
may  have  elapsed  since  the  practice  would  have  been 
implemented  and  any  evidence  of  erosion  or  water  quality 
related  impacts  would  be  difficult  to  assess. 

Teams  addressed  the  timeliness  of  activities  on  sites 
where  an  impact  was  observed  but  could  not  be  attributed  to 
any  other  practice.   In  other  cases,  if  evidence  didn't 
indicate  any  problem  teams  relied  upon  the  memory  of  the 
sale  administrator  or  their  records.   Sometimes  the 
administrator  wasn't  always  the  one  who  actually 
administered  the  sale.   Knowing  only  the  month  or  the  season 
of  practice  completion  doesn't  provide  the  kind  of  answers  a 
question  about  timing  is  trying  to  ask.   It  may  only  tell  us 
in  broad  terms  what  the  potential  risk  of  damage  was.   More 
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specifically,  one  would  want  to  know  more  about  the  specific 
site  conditions,  i.e.,  soil  moisture,  soil  frost,  snowpack 
and/or  climatic  events  experienced  at  the  time  of  the 
activity.   This  approach  was  straying  from  the  intention  of 
"rating  what  we  observe"  and  because  of  this,  items  of  this 
type  were  frequently  not  rated. 

Information  related  to  timing  of  construction  of  road 
drainage  features  and  other  erosion  mitigation  measures  are 
important  and  warrant  inclusion  in  this  process,  but  are  not 
addressed  properly  in  a  post-harvest  review.   To  adequately 
assess  these  practices  and  their  impacts,  one  could  conduct 
a  separate  review  of  these  items  on  timber  sales  in- 
progress. 

If  this  form  is  to  be  used  in  the  future,  items  such  as 
these  should  be  either  removed  from  the  form  or  the 
information  should  be  obtained  from  sources  other  than 
post-harvest  field  reviews. 

Other  items  also  posed  difficulty  in  relation  to  their 
inclusion  on  the  form  or  their  wording.   Generally,  these 
were  related  to  the  question  of  "How  do  team  members 
accurately  assess  forest  practices  on  a  given  site, 
recognizing  the  intent  of  the  BMPs  and  the  diversity  of  site 
conditions,  using  a  numerical  approach?"  This  involves  a 
clear  understanding  of  the  intent  of  the  BMPs,  and  the 
importance  or  contribution  of  a  given  practice  toward 
addressing  that  intent.   Although  that  intent  is  not  always 
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clear  in  the  BMPs  themselves,  structuring  the  audit  form  so 
that  there  is  a  clear  connection  or  cross-referencing 
ability  between  the  form  and  the  BMP  document  would  be  one 
important  improvement . 

In  cases  where  the  intent  can  be  succinctly  stated  on 
the  form  either  through  language  changes  or  additions,  it 
should  be  done.   Care  should  be  exercised  not  to  make  the 
practices  on  the  form  more  restrictive  than  the  wording  in 
the  actual  BMP  itself  or  the  form  will  not  be  measuring  the 
same  thing. 

Several  practices  are  identified  in  the  following 
table,  with  comments  for  improvement.   These  practices 
frequently  presented  difficulties  in  interpretation  for 
ratings. 

Additional  items  suggested  as  possible  improvements  to 
the  audit  form  include: 

1)  The  separation  of  "system  roads"  from  "temporary  roads" 
in  the  rating  process.   In  addition,  the  audit  form 
should  provide  space  for  distinguishing  between  practices 
involving  "new  construction"  and  "re-construction". 

2)  Remove  the  streamside  management  zone  (SMZ)  practices 
from  the  Timber  Harvest  section  and  develop  a  separate 
section  for  all  activities  influencing  the  SMZ,  i.e., 
roads,  harvesting,  slash  disposal,  and  site  preparation. 

3)  Provide  a  separate  section  to  cover  the  practice  of 
broadcast  burning  as  a  method  of  slash  treatment.   Within 
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Table  4.   Existing  Audit  Form  Language  and  Suggestions  For 
Improvement . 


BMP  SECTION 


Construction 


Comments: 


Comments: 


Maintenance 


Comments : 


Maintenance 


Comments: 


PRACTICE 
#  and  DESCRIPTION 

1.  Cut  and  fill  slopes  at  stable  angle 

Wording  might  be  changed,  although  in 
it's  present  form  it  is  almost  verbatim 
to  the  documented  BMP.   Intent  is 
unclear,  could  suggest  room  for  BMP 
language  improvement.   Suggest  reword  to 
"proper  angle"  or  "stabilized". 

6.  Grass  seeding  completed 

Might  make  practice  more  encompassing 
than  just  stabilization  related  to  the 
act  of  seeding.   Rating  this  items 
doesn't  adequately  address  the  complete 
stabilization  of  soil  disturbances 
related  to  road  construction. 

4.  Closed  roads  left  in  condition  to 
provide  adequate  drainage. 

Interpretation  of  the  word  "Closed" 
was  reserved  for  roads  with  barriers  to 
travel,  while  gated  roads  were  considered 
"restricted".   Suggestions  included  the 
use  of  the  word  "abandoned"  instead. 
Needs  a  more-clearly  defined  intent 
regarding  adequate  drainage  to  rate  this 
practice.  After  weathering  for  2  years, 
one  would  expect  less  erosion  than  was 
observed  on  some  and  more  drain  features 
functioning. 

5.  Restrict  use  of  roads  during  wet 
periods  and  spring  breakup 

Interpretation  was  with  respect  to  the 
gating  or  posting  with  signs  to  restrict 
travel.   Rating  teams  should  concentrate 
on  the  need,  and  actual  visible  evidence 
of  the  practice  on  site.   Focus  should  be 
on  efforts  aimed  specifically  at 
addressing  soil  and  water  resources  and 
not  wildlife,  although  they  may  not  be 
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Table  4  (continued) 


mutually  exclusive, 


Timber  Harvest 


Comments : 


Comments : 


3.  Equipment  operation  in  wet  meadows 
and  bogs  avoided 

At  what  point  does  interception  of 
ground  water,  near  the  surface, 
constitute  a  departure?   The  intent  is 
to  prevent  the  development  of  a  more 
efficient  system  of  water  conveyance 
downslope,  although  not  stated  in  the 
BMP.   This  wording  leaves  some  team 
members  thinking  in  terms  of  soil 
displacement. 

4.  Skidding  operation  minimizes  soil 
disturbance 

5.  Skidding  operation  minimizes  soil 
compaction 

For  4  and  5,  teams  were  frequently  not 
certain  of  what  an  acceptable  level  was, 

May  need  guidelines  developed  by  soil 
scientists  as  ratings  were  often 
deferred  to  them. 


Treatment  and 
Site  Preparation 


Comments: 


(.  Operation  done  when  soils  are  dry 
enough  to  minimize  compaction  and 
displacement 

Remove  the  words  "dry  enough"  as  the 
frequency  of  equipment  travel  and  the 
care  in  operating  equipment  can  impact 
even  dry  ground.   Would  still  need 
separate  item  to  address  the  timing 
of  activity,  whether  or  not  it  would  be 
assessed  in  a  post-harvest  BMP  review. 
Provide  guidelines  for  assessing  these 
impacts.   Some  suggest  that  displacement 
and  compaction  should  be  split  out 
separately. 

>.  Protection  of  SMZ  during  slash 
reduction 


Comments:   Add  "and  site  preparation. 


64 


this  category  it  was  proposed  that  the  following  questions 
be  addressed: 

a.  Burn  intensity  too  hot,  resulting  in  soil  erosion 
or  riparian  vegetation  problems  ? 

b.  Was  the  burn  accomplished  in  a  fashion  to  meet 
soil,  water  and  riparian  vegetation  goals  ? 

4)  Provide  a  location  on  the  form  to  identify  the  average 
SMZ  width. 

5)  Change  title  of  section  "Road  Planning  and  Location"  to 
"Road  Planning  and  Design".   Provide  clarity  in  practice 

#6  by  adding  wording  specifically  addressing  the  rolling   of 
the  grade.  Provide  opportunity  to  rate  the  design,  i.e., 
outslope  vs.  insloped/ditched  and  their  effectiveness. 

6)  Remove  the  rating  practice,  "25 •  minimum  SMZ 
maintained?"  from  the  "Timber  Harvest"  rating  section  which 
utilizes  an  ordinal  rating  scale  because  this  question  is 
best   addressed  by  a  yes/no  response.   Some  suggest  removing 
the  item  altogether,  which  would  be  appropriate  if  item 

#4,  above,  were  incorporated  into  the  form. 

Ideally,  the  development  of  a  guidebook  to  compliment 
the  audit  form  would  serve  to  eliminate  some  of  the  problems 
in  BMP  interpretations.   This  could  be  done  with  either  a 
photo  guide  or  a  more  complete  description  of  the  BMP's 
intent.   In  the  absence  of  that,  any  changes  to  the 
practices  on  the  form  should  focus  on  clarifying  the  intent 
of  the  practice  which  teams  are  attempting  to  rate. 

» 
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Other  concerns  regarding  clarification  of  intent  were 
of  a  more  general  nature.    Teams  were  uncertain  of  the 
extent  that  practices  should  be  applied  to  address  soil 
erosion  in  upland  locations.   Some  felt  that  if  erosion  is 
occurring  and  yet  hasn't  reached  a  channel,  we  should  assess 
the  ability  of  the  site  to  stabilize  itself.   If  sediment  is 
being  detached  and  transported  on-site  and  the  probability 
of  it  reaching  the  channel  is  relatively  low,  teams  would 
assign  a  rating  to  the  practice  which  they  felt  was  needed 
to  control  the  erosion.   This  varied  from  receiving  a  H3"  if 
sediment  was  transported  downslope  to  a  "4"  for  no  impact. 
Consistency  in  addressing  upland  soil  erosion  with  this 
process  is  another  area  in  need  of  clarification  with 
respect  to  intent. 

Comments  from  these  audits  can  be  important  indicators 
of  the  need  for  improved  clarity  in  the  BMP  language  to 
improve  the  level  of  understanding  and  application  in  the 
field.   It  was  generally  felt  that  if  the  rating  teams  had 
trouble  interpreting  certain  practices,  field-personnel 
would  have  similar  trouble  implementing  them. 

B.   Rating  Scale: 

The  rating  scale  used  variables  "application"  and 
"effectiveness",  each  with  5  possible  values.  The  following 
section  discusses  comments  and  observations  which  were  made 
concerning  use  of  this  scale  for  assessing  BMPs. 
1.   Is  there  a  need  for  more  precise  rating  values  in  order 
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to  provide  an  objective  assessment  of  practice 
application  and  associated  impacts  ? 

2.  Does  the  existing  rating  scale  influence  the  manner  that 
teams  or  individuals  interact  and  arrive  at  consistent 
ratings,  from  site  to  site,  particularly  in  the  absence 
of  any  documented  procedure  ? 

3.  If  these  values  are  left  loosely  defined  and  open  to 
interpretation,  will  it  significantly  influence  the 
repeatability  of  a  study  of  this  type  and  impair  the 
ability  to  make  comparisons  of  results  between  different 
studies  ? 

4 .  Do  the  results  give  us  the  best  kind  of  data  necessary 
to  improve  the  management  of  our  forest  lands  for  the 
minimization  of  nonpoint  source  pollution  ? 

The  most  frequent  difficulty  encountered  in  using  the  5 
point  scale  was  in  establishing  the  range  of  conditions  or 
observations  that  could  be  associated  with  each  of  the  5 
values.   Even  though  general  guidelines  were  discussed 
during  the  calibration  audit,  the  wide  variety  of  site 
conditions  and  practices  required  a  considerable  amount  of 
interpretation  in  the  field.   Team  members  expressed  a 
desire  to  have  the  scale  of  values  more  rigidly  defined. 
Some  of  these  feelings  were  a  result  of   infamiliarity  with 
the  system,  and  as  time  passed  and  team  members  interacted 
and  experienced  a  variety  of  situations,  they  developed  the 
skills  necessary  to  use  the  system  effectively  and  with 
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improved  consistency.  Unfortunately,  much  of  the  field 
season  passed  before  these  skills  were  successfully 
developed. 

Although  a  system  which  removes  all  professional 
judgement  is  not  the  answer,  there  are  improvements  which 
can  be  made  to  remove  some  of  the  subjectivity  of  this 
system  and  improve  the  consistency  of  applying  this 
methodology. 

The  language  used  in  the  rating  value  descriptions  was 
a  source  of  some  difficulty.   The  language  uses  negative 
connotations  that  can  be  somewhat  threatening  for  some 
people.   Phrases  such  as  "major"  and  "minor"  departures  may 
be  fine  in  a  regulatory  environment  where  practice 
compliance  is  the  primary  objective.   Under  the  cooperative 
framework  they  may  act  as  a  barrier  to  assigning  rating 
values  which  accurately  depict  the  practice.   Each  of  these 
terms  have  meanings  which  may  be  taken  out  of  context  and 
may  cause  individuals  to  shy  away  from  using  a  number  which 
contains  such  language,  for  fear  of  misuse  of  results. 
Attempts  should  be  made  to  use  terminology  which  does  not 
carry  such  negative  connotations,  but  instead  points  toward 
a  more  accurate  description  of  the  level  of  BMP  application 
or  effectiveness. 

Another  problem  with  the  rating  language  is  that  no  one 
can  precisely  define  what  constitutes  a  "major"  or  "minor" 
departure  or  impact.   Team  members  develop  mental  pictures, 
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even  though  frequent  reminders  were  given  during  team 
discussions  to  maintain  a  level  of  consistent 
interpretation.   When  an  individual  team  member  took  a 
position  that  either  a  "major"  or  "minor"  departure  or 
impact  had  occurred,  they  sometimes  found  it  difficult  to 
support  that  opinion.   Many  times  the  burden  of  proof  rested 
solely  on  the  professional  opinion  of  an  individual  during  a 
discussion  because  no  definition  existed  for  the  terms 
"major"  or  "minor".    In  fact,  impacts  may  have  occurred  but 
have  since  been  masked  by  the  passage  of  time,  leaving 
little  evidence  to  support  one's  position. 

Once  team  members  were  challenged  on  their  opinions, 
they  may  not  have  been  confident  in  their  convictions,  were 
placed  in  the  minority  or  persuaded  to  change  to  a  rating 
value  which  was  less  controversial.   This  was  usually  not 
the  case  when  impacts  were  either  still  occurring  or  were  so 
extensive  as  to  warrant  the  use  of  the  terms  without 
hesitation.   The  terminology  and  their  connotations  and  the 
lack  of  a  clear  distinction  between  the  two  terms  tends  to 
discourage  individual  participation.   This  was  demonstrated 
by  the  time  it  took,  on  some  teams,  for  someone  to  "stick 
their  neck  out"  and  put  a  rating  up  for  discussion.   This 
was  most  notable  on  sites  where  it  was  more  obvious  that 
departures  in  BMPs  existed  and  the  potential  for 
considerable  discussion  was  evident. 


69 


One  alternative  would  be  to  remove  the  language  that 
doesn't  serve  the  interaction  process  well.   Suggestions  for 
language  changes  on  the  application  scale  include  replacing 
wording  such  as  "minor  departure  from  intent  of  BMP"  with 
"alternative  practice  or  standards  desired  to  mitigate 
erosion."  and  replacing  "major  departure"  with  "absence  of 
necessary  practices  to  mitigate  erosion".   It  was  also 
proposed  that  the  rating  for  application  termed  "gross 
neglect  of  BMPs"  be  dropped  from  the  form  as  it  provides  no 
additional  value  beyond  the  other  ratings  which  are 
available. 

Similar  descriptive  language  could  be  developed  for  the 
"effectiveness"  scale  that  would  better  define  short-  and 
long-term  and  clarify  what  might  constitute  a 
"minor/temporary  impact",  and  "major  impact". 

Once  the  language  was  modified  to  a  more  functional 
level  for  interpretation  in  the  field,  a  methodology  should 
be  developed  for  a  more  systematic  approach  to  the 
assignment  of  a  rating.   This  could  be  a  simple  outline  of 
the  necessary  steps  in  the  thought  and  observation  process 
that  are  necessary  to  arrive  at  a  rating. 

The  following  list  provides  some  guestions  and 
considerations  for  such  a  methodology: 
1.   Do  site  conditions  and  silvicultural  prescription 

warrant  the  application  of  a  certain  BMP  ? 
a.   Are  practices  being  implemented  ? 
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b.   What  are  the  limitations  to  employing  a  practice  ? 

Considerations  include: 

af.      Physical  limitations  of  the  site 

b.  Silvicultural  objectives 

c.  Equipment  limitations 

d.  Economics 

2.  What  is  the  quality  of  BMP  implementation  ? 
Considerations  include: 

a.  Design  standard  or  level  of  execution 

b.  Consistency  of  application 

c.  Frequency  of  application 

3.  Are  BMPs  minimizing  soil  and  water  impacts  ? 

a.  Is  the  observed  soil  disturbance  associated  with 
timber  harvesting  or  related  activities  ? 

b.  Attempt  to  quantify  the  area  of  disturbance  focusing 
how  much  is  localized  vs.  distributed  over  broad 
areas. 

c.  Is  sediment  reaching  a  stream  channel  ? 

d.  Has  sediment  been  flushed  off  site  as  a  result  of 
the  practice  leaving  an  erosion  pavement? 

e.  Will  sediment  reach  a  channel  or  leave  the  site  in 
the  future  ? 

f.  How  long  will  the  observed  erosion  or  the  potential 
for  erosion  continue  before  stabilization  occurs  ? 

g.  What  beneficial  uses  will  be  impacted  by  increases 
in  sediment  in  a  given  stream  reach  ? 
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4.   Are  there  questions  regarding  BMP  interpretation  and 
needs  for  revision  ?  Considerations  include: 

a.  Clarity  in  the  intent  of  the  BMP 

b.  Adjustment  of  BMP  language  to  reflect  practices  on 
difficult  site  types  i.e.,  granodiorite  vs. 
metasediment 

These  were  questions  that  each  team  member  answered  at 
some  point  during  the  field  audits  but  for  which  the  form 
and  rating  scale  proved  inadequate  to  address  in  a  direct 
and  consistent  manner. 

A  broader  question  which  might  be  asked  for  future 
studies  is  "Does  this  rating  scale,  and  associated 
descriptive  provide  the  kind  of  data  which  is  easily 
interpreted  and  most  useful  to  assessing  non-point  impacts 
from  forestry  practices  ?"  Answers  to  this  question  might 
point  to  a  need  to  a  complete  redesign  of  the  ratings  scale 
and  procedures  for  assessing  practices.   Decisions  regarding 
this  should  balance  the  importance  of  linking  the  data  from 
future  studies  to  those  of  the  past  against  the  potential 
benefits  of  gathering  information  which  may  be  more  valuable 
in  describing  watershed  condition  and  nonpoint  source 
pollution. 

Instead  of  teams  becoming  locked  up  over 
interpretations  of  "major"  vs.  "minor",  which  occurred 
frequently,  an  improved  field  form  might  provide  the 
direction  necessary  to  consistently  apply  this  methodology. 
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Such  an  improvement  might  lead  to  more  meaningful  and 
consistent  data  from  site  to  site. 

Another  alternative  rating  guide  for  practice 
"application"  would  drop  the  "major"  and  "minor"  distinction 
in  favor  of  the  following  approach. 

1.  Is  the  practice  necessary  on  this  site?   Yes/No 

2.  Was  the  practice  adeguately  applied  to  mitigate  erosion 
and  protect  water  guality  ?  Yes/No 

3.  If  No  to  Question  2,  then  check  the  appropriate  box 
below  that  represents  the  approximate  freguency  or  extent  of 
the  practice  departure  on-site? 

0  -  10  % 

10  -  40  % 

40  -  70  % 

70  -  100  % 

With  these  specific  guest ions  to  answer,  team  members 
will  not  only  have  a  more  systematic  approach  to  assigning  a 
rating  by  avoiding  the  connotation  associated  with  "loaded" 
terms,  but  will  also  improve  consistency  in  ratings  from 
site  to  site.   Further,  these  data  lend  themselves  to 
statistical  freguency  analysis. 

It  is  difficult  to  assess  what  impact  rating 
assignments  had  on  the  outcome  of  the  audits,  or  whether 
ratings  might  have  been  significantly  different. 

The  recommendation  of  many  was  to  put  the  results  from 
the  rating  scale  into  two  groups  -  those  practices  which 
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were  adequately  applied  and  effective  and  those  which  were 
not.  If  this  were  to  be  the  extent  of  the  analysis,  then 
efforts  at  refining  a  scale  to  provide  distinction  between  5 
rating  values  seem  to  be  academic. 

Efforts  to  standardize  both  the  form  and  methodology 
should  continue,  as  it  appears  that  these  assessments  will 
become  one  vehicle  for  controlling  non-point  source 
pollution  from  forested  lands  in  Montana. 

Although  we  want  to  link  the  results  of  this  study  to 
previous  efforts,  the  importance  of  the  development  of  such 
a  tool  which  will  perform  over  the  long-term,  should  not  be 
underestimated.   The  system  should  be  flexible  enough  to 
undergo  changes  as  they  become  necessary  while  providing 
direct  and  consistent  information  to  answer  questions 
regarding  the  effectiveness  of  BMPs  in  controlling  nonpoint 
source  sediment  production. 
5.   Field  Work 

The  field  work  began  with  providing  training  in  the 
uses  of  the  assessment  tool.   While  the  "calibration  audit" 
was  a  good  opportunity  to  get  out  on  the  ground  and  apply 
the  methodology,  logistically  it  could  not  provide  the  kind 
of  exposure  to  site  variability  which  teams  encountered 
during  the  field  season.   Nevertheless,  exposure  was 
necessary  for  improving  the  consistency  of  ratings  from 
site-to-site.   A  slide  presentation  that  captures  some  of 
this  variation  might  be  very  valuable.   The  slides  could  be 
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viewed  and  discussed  by  audit  teams  to  establish  a  framework 
for  the  uniform  assignment  of  ratings. 

On  site,  it  was  very  important  to  cover  as  much  ground 
as  possible  guickly.   Any  time  saving  short-cuts  resulted  in 
short-changing  the  thorough  review  of  all  practices. 
However,  a  suggestion  for  improvement  in  this  area  was  to 
provide  the  ownership  representative  a  list  of  general  audit 
concerns  so  that  he/she  may  direct  teams  to  areas  where 
these  practices  occurred. 

Although  teams  were  successful  in  covering  3  sales  in 
one  day,  team  members  felt  that  2  sales  might  have  been  a 
more  realistic  objective.   Understandably,  the  effectiveness 
of  their  work  and  attention  to  detail  declined  in  the  late 
afternoon  and  early  evening. 
6.   Consistency  in  Ratings 

Attempt  was  made  to  quantify  the  level  of  consistency 
in  ratings  among  teams.   BMP  assessments  on  the  same  3  sales 
were  conducted  by  all  three  teams  at  different  times 
throughout  the  field  season.   The  3  sales  offered  wide 
ranges  of  BMP  requirements,  application  and  effectiveness. 
Where  BMPs  were  applied  and  effective,  there  was  a  high 
level  of  consistency  in  ratings  among  teams.   Where  levels 
of  BMP  application  were  less  than  adequate,  consistency  was 
also  good. 

The  greatest  differences  in  ratings  occurred  on  sites 
that  impacts  were  not  as  easily  linked  to  specific 
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departures  from  BMPs.   For  example,  one  team  felt  that  the 
road  drainage  was  responsible  for  concentrating  water  and 
energy  and  contributing  to  surface  erosion,  where  another 
team  viewed  timber  harvest  activities  as  contributing  a 
greater  impact.   Both  teams  may  have  rated  road  drainage 
and  timber  harvest  practices  less  than  adequate,  but  the 
ratings  for  each  of  the  practices  were  different. 

For  the  comparison  sites  only,  the  number  of  rating 
values  were  reduced  from  5  to  2.   All  scores  of  4  or  5  were 
lumped  together  (adequate  and  effective  or  better)  and  all 
scores  of  3 ,  2  or  1  were  lumped  together  (inadequate  and 
ineffective  or  worse) .  Table  4  illustrates  team  comparisons 
in  ratings  across  the  same  3  timber  sales. 

Using  the  simpler  yes-or-no  rating  scheme,  teams  were 
actually  fairly  consistent.   Teams  2  and  3  were  very  similar 
in  their  ratings  on  the  Napa  Goat  and  Private/Sinclair  Creek 
sales.   The  disagreement  between  team  2  and  team  3  on  the 
Lower  Tamarack  sale  was  because  team  2's  misdirection  on- 
site  that  resulted  in  their  not  seeing  a  road  used  during 
harvest  and  eroding  badly  because  of  poor  drainage.  Team  1 
consistently  scored  the  same  sales  more  favorably  than  the 
other  two  teams. 

The  consistency  seem  among  teams  using  the  2-value 
rating  scale  on  the  comparison  sites  might  be  a  legitimate 
argument  for  using  the  2-value  scale  for  the  entire  audit 
process.   Again,  a  team  or  person  is  usually  able  to  judge 
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whether  a  practice  is  adequate  or  inadequate,  but  it  is  not 
so  easy  for  2  individuals  to  agree  upon  the  magnitude  of  the 
departure  or  impact. 

Another  question  of  consistency  involved  that  which 
occurs  between  sites.   This  involves  the  question  of  "Does  a 
rating  of  a  3  on  one  practice  on  one  sale  carry  the  same 
meaning  as  a  3  on  the  same  practice  on  a  different  sale  ?" 
Although  frequently   similar,  observations  indicated  it  was 
not  always  the  case.   There  seemed  to  be  too  many  variables 
influencing  a  rating  at  a  given  point  in  time.   The 
susceptibility  of  ratings  to  individual  interpretation, 
although  tempered  by  the  interaction  by  the  group  and 
moderator,  made  placing  practices  from  two  different  sales 
on  the  same  rating  scale  difficult.   These  difficulties  may 
point  out  subtle  areas  where  further  refinements  in  BMP 
language  may  be  beneficial. 
7.   Other  Influences  in  Ratings 

Many  factors  influenced  the  final  rating  value  assigned 
to  a  practice.   The  use  of  interdisciplinary  teams  to 
conduct  these  audits  brings  together  a  diverse  group  of 
individuals  with  different  understanding  and  interpretation 
of  the  BMPs.   The  dynamics  of  group  interaction  and  the 
communication  skills  of  individuals  within  the  group  can 
effect  the  rating  values.  Not  all  audit  team  members 
consistently  took  a  professional  and  objective  approach  to 
the  process.   Most  displayed  consistency  in  their  concerns 
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for  the  water  and  soil  resources  and  that  was  reflected  in 
the  discussions  of  the  ratings,  regardless  of  the  ownership 
evaluated.   Opinions  were  not  always  volunteered  freely.   At 
times  it  was  necessary  to  question  team  members  individually 
to  obtain  his/her  rating. 

The  performance  of  the  interdisciplinary  teams  in  the 
assignment  of  ratings  varied  at  times  but  was  generally 
regarded  as  very  professional.   Ratings  were  not  influenced 
at  the  decision  level  where  it  was  necessary  to  establish 
the  "need"  for  a  practice.   Nor  did  teams  generally  have  any 
difficulties  in  deciding  if  practices  were  applied  or 
whether  impacts  had  occurred.   Problems  in  reaching  a 
consensus  were  related  to  the  "magnitude"  of  the  departure 
and  the  impacts. 

These  differences  and  the  eventual  rating  were  a 
function  of  site  conditions,  the  obvious  nature  of  an 
impact,  the  rating  scale  used,  the  differences  in  perception 
of  what  needed  protection,  and  the  persuasiveness  and 
determination  of  individuals  in  the  group  during 
discussions.   These  observations  indicate  that  the 
reliability  of  the  ratings  are  strongest  when  the  5  point 
rating  scale  is  re-classified  into  two  categories  for  each 
variable: 

Application:    1.  practices  applied 

2 .  practice  not  applied 
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Effectiveness:  1.  adequate  resource  protection 

2.  inadequate  resource  protection. 

Age  of  the  sale  being  reviewed  may  also  influence 
ratings.   Our  study  allowed  for  the  examination  of 
activities  which  occurred  over  previous  years.   After 
looking  at  a  variety  of  different-  aged  sales  the  teams  felt 
that  age  can  have  a  significant  influence  on  a  team's 
ability  to  accurately  assess  the  impacts  resulting  from 
management  activities.   Depending  on  the  moisture  conditions 
of  the  site,  varying  densities  of  vegetative  cover  can 
hamper  efforts  to  identify  impacts.   The  first  runoff 
following  the  management  activity  is  generally  recognized  as 
the  most  critical  time  for  sediment  production.   Any 
observation  conducted  on  a  site  with  activities  older  than  1 
year  may  result  in  missing  some  of  the  observable  impacts. 
Some  sales  were  considerably  older  than  others  and  may  not 
have  received  the  same  rating  if  audited  nearer  to  sale 
completion. 

Grazing  impacts  had  the  potential  to  influence  a 
rating.  Many  of  the  sites  had  streambanks  that  were  heavily 
impacted  by  cattle.   Team  members  felt  that  an  adequate 
distinction  could  be  made  between  impacts  associated  with 
harvesting  timber  from  those  related  to  grazing.   On  sites 
where  grazing  impacts  were  not  as  severe,  teams  inspected 
the  channel  for  evidence  of  sedimentation  which  was  probably 
related  to  harvesting. 
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Table  5.  Team  comparisons  of  ratings. 

t  PERCENTAGES  OF  PRACTICES 


% 
< 

NAME: 

TEAM  # 
1 
2 
3 

APPLICATION 

* 
* 

* 
* 

* 

* 

EFFECTIVENESS 

RATING1    %  RATING 
A  SCORE     >  OR  =  4 
OF  4 

% 
< 

RATING 
A  SCORE 
OF  4 

% 
> 

RATING 
OR  =  4 

SALE 

NAPA  GOAT  #4 

16          84 
32          68 
32          68 

16 
32 
26 

84 
68 
74 

SALE 

NAME: 

TEAM  # 
1 
2 
3 

LOWER  TAMARACK 

16          84 

4          96 

20          80 

* 
* 
* 

8 

0 

12 

92 

100 

88 

SALE 

NAME: 

TEAM  # 
1 
2 
3 

PRIVATE/ SINCLAIR 

54          46 
69          31 
69          31 

CREEK 

* 

* 

54 
72 
66 

46 
28 
34 

The  number  4  refers  to  the  rating  scale  value,  and 
represents  an  adequate  level  of  practice  application  and 
effectiveness . 

CONCLUSIONS 

Future  audits  should  review  the  site  selection  criteria 

for  possible  improvements  in  site  sampling.   This  should 

include  consideration  of  "new  road  construction"  as  a  higher 

priority  that  "slash  treatment  completed"  when  establishing 

site  selection  criteria.   This  is  because  of  both  it's 

greater  capacity  to  influence  water  quality  and  the  higher 

proportion  of  practices  which  it  represents  on  the  audit 
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process  form.   Also  needed  are  adjustments  in  the  "window" 
of  time  between  completion  of  sale  and  practice  review  to 
capture  the  time  period  representing  the  greatest  risk  and 
most  observable  impacts  to  water  quality.   Other  changes 
might  include:  field  review  of  practices  immediately 
following  the  first  runoff  season,  or  "in-process"  reviews; 
if  "slash  completion"  remains  a  criteria  for  selection  then 
slash  backlog  should  be  reduced  to  some  acceptable  level 
before  audits  are  conducted  to  insure  a  more  complete  "pool" 
from  which  to  sample;  using  the  silvicultural  prescription 
to  establish  the  intensity  of  harvest  activity  vs.  the 
volume  of  timber  removed  per  acre. 

Although  teams  had  difficulty  on  some  sites  with  the 
BMP  language  associated  with  certain  practices,  in  general 
it  seemed  sufficient  to  carry  out  these  assessments.   Some 
practices  require  more  specifically  defined  intents  so  that 
foresters  and  equipment  operators  can  understand  why  it  is 
important  to  apply  a  practice.   Instead  of  moving  toward 
changes  in  the  practices  we  have,  we  might  first  focus  our 
efforts  on  education.   For  example,  education  regarding 
adequate  drainage  for  open  and  closed  roads  and  adequate 
number  of  drainage  relief  features  for  roads  on  specific 
soil  types  can  provide  improvements  in  BMP  interpretation. 

There  are  practices  that  could  use  elaboration  in 
identifying   intent.  Specific  problems  with  interpretation 
were  encountered  in  the  area  of  soil  compaction  and 
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displacement.   Guidelines  are  necessary,  not  only  for  these 
assessments  but  for  the  operator  and  administrators  to  apply 
these  practices  within  the  intent  of  the  practice. 

This  methodology  is  in  it's  infancy.   Additional  work 
is  needed  to  develope  the  BMP  assessments  in  the  areas  of: 
improving  training  of  field  teams  in  recognizing  BMP 
departures  and  impacts  prior  to  going  on-site;  modifying  the 
field  audit  form  to  consider  only  those  practices  which  can 
be  visually  inspected  in  the  field;  provide 
cross-referencing  of  practices  on  the  field  form  to  the 
State's  BMP  document;  change  the  rating  scale  form  and/or 
language  and  provide  a  guideline  for  field  teams  to 
distinguish  among  values,  establishing  in-process  audits  for 
those  important  practices  that  relate  to  timing  of 
activities. 

RESULTS 

This  study  was  designed  to  address  a  number  of 
questions: 

1.  What  is  the  current  level  of  BMP  application  in 
the  Flathead  Basin  and  how  effective  have  those 
practices  been  in  preventing  sediment  from  entering 
stream  channels? 

2 .  How  do  the  levels  of  BMP  application  and 
effectiveness  compare  with  the  results  of  the  1988 
Environmental  Quality  Council's  BMP  study? 

3.  What  specific  categories  of  BMP  implementation  and 
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practices  are  contributing  the  greatest  number  of 
impacts?   Where  do  improvements  need  to  be  made? 

4.  How  does  BMP  implementation  vary  by  ownership? 

5 .  New  BMPs  provide  for  greater  protection  of  the 
streamside  management  zone.   Even  though  not 
officially  considered  in  this  audit,  how  would 
those  practices  have  been  rated  if  they  were? 

Three  teams  audited  a  total  of  52  timber  sales 
throughout  the  Flathead  River  Basin.   Each  team  had  an 
opportunity  to  assess  practices  and  impacts  across  the 
entire  basin. 

Each  timber  sale  audited  potentially  had  39  separate 
management  practices  to  evaluate.   Each  practice  was  rated 
for  the  degree  to  which  it  had  been  applied  and  the  degree 
to  which  the  practice  contributed  sediment  or  increased  the 
likelihood  of  sediment  reaching  surface  waters. 

Not  all  practices  were  applicable  on  every  sale.   In 
total,  1427  practices  were  evaluated  for  the  application  of 
BMPs  on  52  timber  sales,  which  is  approximately  27  practices 
per  sale. 

BMP  PERFORMANCE  IN  THE  FLATHEAD  BASIN 

Four  methods  of  analyzing  the  findings  of  these  audits 
have  been  selected  to  address  the  first  guestion: 

1)  Analysis  of  BMP  application  and  effectiveness  by 
percentages  of  the  "total  practices  audited" . 

2)  Analysis  of  the  percentage  of  practices  audited  on 
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each  ownership  which  are  contributing  to  major  and 

minor  impacts. 
3)   Analysis  of  percentages  of  "timber  sales" 

contributing  to  impacts. 
Method  1  -  Analysis  by  "Total  Practices  Audited" 

About  2  percent  of  the  1427  practices  exceeded  the 
requirements  of  the  BMPs;  88  percent  met  the  BMPs;  7  percent 
were  rated  as  minor  departures;  and  2  percent  were  rated  as 
major  departures  and  1  percent  were  considered  "gross 
neglect". 

Similarly,  1427  practices  were  rated  for  effectiveness. 
Of  these,  less  than  1  percent  were  considered  practices 
which  improved  resource  protection  over  pre-project 
conditions;  91  percent  were  rated  as  adequate  resource 
protection;  7  percent  rated  as  causing  minor  or  temporary 
detrimental  impacts;  2  percent  were  rated  as  causing  major 
detrimental  impacts,   which  were  primarily  short  term;  and 
less  than  1  percent  were  evaluated  as  major  detrimental 
impact,  with  extensive  damage  and  long-term  recovery. 

Table  6  provides  a  summary,  by  individual  practice,  of 
the  rating  numbers  assigned  for  each  ownership.   Appendix  A 
provides  a  copy  of  the  field  rating  form  with  the  guide  used 
to  assign  the  values  from  1  to  5. 
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Table      6.      Summary  of   BMP    ratings 


5PPLICATIQN 


EFFECTIVENESS 


BMP         OWNERSHIP  12  3  4: 

nuiuiiUMUi  summit  uuu  mm  nun  Mint  mm 

-GADS  -  PLANNING 

i.  MINIMIZE 

4  GF  ROADS      NON-IND.PRIV  0  0  0           5           0 

PLLtfl  CREEK  0  0  0          13            1 

STATE  LANDS  0  0  0            4            0 

iiSFS  0  0  0         22          0 

nmt  tutu  tutu  tuut  tuut 

TOTAL  0  0  0         49           i 


iitili  utut  aim  tuut  iimi 


0  0  0  5  0 

0  0  0  19  0 

i)  0  0  4  0 

0  0  0  22  0 

utut  ttttu  turn  utut  tttttt 

0     0     0    50     0 


2.  USE  EXIST. 

ROADS 

hON-IND.PRIV 

o 

o 

1 

i 

3 

0 

o 

o 

1 

7 

0 

PLUM  CREEK 

p. 

1 

1 

15 

o 

0 

1 

1 

15 

0 

STATE  LANDS 

0 

o 

o 

b 

l) 

0 

0 

0 

6 

0 

USFS 

'') 

ll 

0 

12 

0 

o 

o 

0 

12 

0 

ttttu 

tutu 

tttttt 

tttttt 

tttttt 

tttttt 

ttttu 

tttttt 

tttttt 

ttttu 

TOTAL 

'.' 

1 

L 

36 

0 

o 

1 

L 

36 

0 

3.  APf'ROP.  ROAD 

STANDARDS 

NQN-IND.PRIV. 

o 

1 

o 

a 

o 

0 

1 

o 

L 

0 

PLUM  CREEK 

0 

1 

o 

12 

o 

0 

1 

0 

12 

0 

STATE  LANDS 

0 

o 

0 

4 

0 

0 

o 

0 

4 

o 

USFS 

(i 

A 

o 

14 

o 

0 

o 

1 

15 

0 

ttttti 

ttttu 

tttttt 

Uttt* 

ttttu 

tttttt 

tttttt 

tttttt 

tttttt 

tttttt 

TOTAL 

0 

1 

o 

34 

o 

0 

1 
L 

1 

33 

0 

4.  L0CATICN  TO 

AVOID  HAZARD 

N0N-IND.PRIV 

o 

0 

1 

1 

o 

0 

0 

0 

n 

L 

0 

PLUH  CREEK 

o 

o 

1 

11 

o 

0 

0 

0 

12 

0 

STATE  LANDS 

o 

o 

0 

3 

0 

0 

0 

1 

H 

L 

o 

USFS 

0 

o 

0 

11 

o 

o 

0 

0 

11 

0 

TOTAL 


(ttttt  tttttt  tttttt  tttttt  tttttt 

0     0     2    26     0 


tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     1    27     0 


5.  ADEQUATE  SMZ 

FROVIDED   NON-IND.FRIV 
PLUM  CREEK 
STATE  LANDS 
USFS 

TOTAL 


0  0 
0  1 
0  0 
0     0 

tutu  tttttt  tttttt  tttttt  utut 
0     1     1    19     0 


0 

1 

0 

o 

0 

o 

ij 

3 

o 

o 

1 

0 

o 

3 

(1 

0 

0 

0 

1 

7 

o 

o 

0 

0 

8 
tttttt  tttttt  tttttt  tttttt  tttttt 

0     1     0    20     0 
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Table      6.      (continued) 


APPLICATION 


EFFECTIVENESS 


BilP         OWNERSHIP  12  3*5 

ttmtttmut  luimnui   mm  mm  mm  mm  mm 

6.  PERMITS  FOR 


12  3  4  5 

mm  turn  mm  mm  mm 


CROSSINGS 

NON-IND.PRIV 

0 

0 

0 

0 

0 

0 

1 

0 

0 

PLUM  CREEK 

o 

0 

4 

0 

0 

o 

0 

4 

0 

STATE  LANDS 

o 

0 

') 

o 

0 

0 

0 

5 

i. 

0 

USFS 

o 

o 

7 

o 

0 

o 

o 

i 

o 

mm 

turn 

mm 

mm 

tmu 

mm 

mm 

mm 

mm 

mm 

TOTAL 

o 

0 

13 

o 

0 

0 

l 

13 

0 

7.  AVOID  LONG 

STEEP  GRADES 

NON-IND.PRIV 

0 

o 

i 

1 

0 

0 

o 

l 

1 

o 

PLUM  CREEK 

o 

0 

0 

13 

0 

0 

o 

o 

13 

o 

STATE  LANDS 

o 

o 

0 

4 

o 

0 

0 

0 

4 

0 

USFS 

c 

0 

0 

11 

o 

o 

o 

0 

11 

o 

TOTAL 


mm  turn  mm  mm  mm 

0  0  1  29  0 


mm  mm  mm  mm  mm 

0  0  1         29  0 


3.  MINIMIZE  # 

OF  XINGS   NON-IND.PRIV 

0 

0 

1 

1 

o 

PLUM  CREEK 

o 

0 

o 

10 

1 

STATE  LANDS 

0 

0 

0 

4 

0 

USFS 

0 

0 

1 

11 

0 

TOTAL 


niiii  mm  mm  mm  mm 

0  0  2         26  1 


0  0  11  0 

0  0  0         11  0 

0  0  0  4  0 

0  0  0         12  0 

mm  mm  mm  mm  mm 

0  0  1         29  0 


ROAD  -  DRAINAGE 

1.  ADES. SURFACE 

DRAINAGE   NON-IND.PRIV 

1 

0 

1 

1 

1 

0 

o 

1 

1 

1 

0 

PLUM  CREEK- 

0 

1 

3 

3 

0 

0 

1 

6 

10 

0 

STATE  LANDS 

0 

0 

1 

5 

o 

0 

0 

5 

4 

0 

USFS 

o 

0 

4 

13 

o 

0 

1 

i 

11 

1 

TOTAL 


mm  mm  mm  mm  mm 

1  1  14  27  0 


unit  mm  mm  mm  mm 

0  3         13         25  1 


2.  TIMELY 

INSTALLATION  NON-IND.PRIV 

1 

0     1 

0    0 

PLUM  CREEK 

0 

0     1 

12     0 

STATE  LANDS 

o 

0     0 

4     0 

USFS 

0 

0     0 

16     0 

mm 

mm  mm 

mm  mm 

TOTAL 

i 

32     0 

0 

1 

1 

0 

0 

0 

o 

1 

12 

o 

0 

0 

0 

4 

o 

0 

o 

1 

15 

o 

mm  mm  mm  mm  mm 

0  1  3         31  0 
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Table     6.      (continued) 


APPLICATION 


EFFECTIVENESS 


BMP         OWNERSHIP  12  3  4  5 

uttuutuutu  tmmmt    mm  mm  mm  mm  tmu 

ROADS  -  DRAINAGE 


12  3  4  5 

licit  mm  mm  mm  nun 


3.  DRAINAGE 

THRU  SH2 

NON-IND.PRIV 

0 

o 

0 

0 

0 

PLL'rt  CREEK 

o 

1 

3 

-' 

■'l 

V 

STATE  LANDS 

0 

0 

0 

7 

o 

USFS 

0 

1 

3 

i 

o 

mm 

lUtlt 

utm 

unit 

mm 

TOTAL 

0 

5 

t> 

12 

o 

*.  PROPER  XING 

INSTALLATION  NGN-IND.PRIV 

o 

0 

1 

1 

o 

PLUM  CREEK 

o 

0 

2 

5 

0 

STATE  LANDS 

o 

o 

0 

n 

i 

USFS 

o 

1 

0 

7 

1 

TOTAL 


mm  mm  mm  mm  mm 

0  1  3         15  2 


0  0  0  0  0 

0  13  3  0 

0  0  0  3  0 

o        o        2        e        o 
tttttt  utm  mm  tttttt  tutu 

0  1  5  14  0 


o 

0 

1 

1 

1           0 

0 

c 

7 
L 

5           0 

o 

u 

n 

3           0 

o 

i 

i 

0 

a        o 

mm 

tutu 

tutu 

tutu  tutu 

0 

1 

1 

g 

17           0 

ROADS  -  CONSTR. 

1.  STABLE  CUT  fe 

FILL  SLOPES    NON-IND.PRIV 

0 

0 

1 

i 

0 

o 

PLUM  CREEK 

0 

0 

1 

12 

o 

STATE  LANDS 

0 

0 

0 

4 

0 

USFS 

0 

0 

o 

.  7 

Li 

o 

TOTAL 


mm  mm  tutu  tttttt  mm 

0     0     2    28     0 


0  0  1     0  0 

0  0  2    11  0 

0  0  13  0 

0  0  1    11  0 

tttttt  tttttt  tttttt  tttttt  tttttt 

0  0  5    25  0 


2.  HALT  WHEN 

WET 

NON-IND.PRIV 

0 

o 

o 

o 

o 

PLUM  CREEK 

0 

0 

o 

11 

o 

STATE  LANDS 

o 

0 

o 

4 

o 

USFS 

0 

0 

0 

14 

0 

TOTAL 


tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     0    2?     0 


0  0  0  0  0 

0  0  0  11  0 

0  0  0  4  0 

0  0  0  14  0 

tttttt  tttttt  tttttt  tttttt  tttttt 

0  0  0  2'  0 


3.  EROSION 

CONTROL  KEPT  NON-INDPRIV 
CURRENT   PLUM  CREEK 
STATE  LANDS 
USFS 

TOTAL 


1 

0 

0 

0 

0 

0 

1 

0 

? 

0 

0 

o 

1 

3 

0 

o 

0 

0 

14 

0 

tttttt  tttttt  tttttt  tttttt  tttttt 

1  1  1         2i  0 


0  10  0  0 

0  10  9  0 

0  0           0  4  0 

0  0           0  14  0 

tttttt  tttttt  tttttt  tttttt  tttttt 

0  2           0  27  0 
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Table   6.         (continued) 


APPLICATION  EFFECTIVENESS 


BflP         OWNERSHIP  1           Z           :           4           5  i           2           3           <1           5 

utttuttutttt  uttuutut  tmtt  iittn  mm  tutu  inn:  mm  mm  mm  mm  tuiti 

4.  CLEAR  VEB. 

FRQH  FILL   NON-IND.FRIV  0     0     0     I     0  0     0     i     0 

PLUM  CREEK  0     0     0    II     0  0     0     0    13 

STATE  LANDS  0     0     0     *     0  0     0     0     4     0 

USFS  0     0     0    13     0  0     0     0    13     0 

tutu  sum  tutu  tttttt  utut  tutu  tutu  turn  ttuti  tmtt 

TOTAL  0           0           0         31           0  0           0           0         31           0 


5.  OVERBURDEN 

PLACEMENT   NCN-IND.PRIV 

o 

o 

0 

o 

o 

PLUM  CREEK 

0 

0 

0 

6 

o 

STATE  LANDS 

0 

0 

o 

1 

L 

0 

USFS 

o 

0 

o 

9 

o 

'.'  ,'  'J  a  •.! 

0  0  0  2  0 

i)  0  0  9  0 

mtti  tmtt  nuu  utm  mm  turn  mm  mm  tttttt  turn 

TOTAL  0  0  0  17  0  0  0  0  17  0 


6.  GRASS 

" 

SEEDIN6 

NON-IND.PRIV 

1 

0 

0 

0 

0 

1 

0 

o 

o 

0 

PLUM  CREEK 

0 

0 

2 

? 

0 

0 

o 

S 

-? 

0 

STATE  LANDS 

ft 

o 

0 

3 

o 

0 

g 

o 

7 

0 

USFS 

0 

0 

o 

14 

0 

0 

0 

J 

12 

j 

tutu  turn  tutu  tttttt  uuu  tutu  tttttt  utut  ttutt  utttt 

TOTAL          1  0     2  26     0  1     0     6    22     0 

ROADS  - 
MAINTENANCE 

1.  ROAD  GRADINS  NON-IND.PRIV             10  0  10  0           10           10 

PLUM  CREEK                0  1           4  13           0  0           0           3          15           0 

STATE  LANDS               0  0           0  5           0  0           0           0           5           0 

USFS                           0  0           2  16           0  0           1           1          16           0 

tttttt  tttttt  tttttt  utut  tttttt  utttt  tttttt  tttttt  ttutt  tttttt 

TOTAL                         1  1           6  35           0  0           2           4         37           0 


2.  FUNCTIONAL 

CULV. /DITCH.  NON-IND.PRIV 

0 

o 

1 

o 

0 

0 

o 

1 

o 

0 

PLUM  CREEK 

o 

•) 

3 

3 

o 

0 

1 

4 

8 

o 

STATE  LANDS 

o 

o 

i 

c 

o 

0 

0 

t 

L 

5 

o 

USFS 

o 

0 

o 

16 

o 

0 

l'i 

I 

15 

o 

utttt  tttttt  tttttt  mm  tutu  utttt  tttttt  ttutt  ttutt  tutu 

TOTAL  0  2  5  2?  0  0  1  7         28  0 
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Table      6.      (continued) 


hPPLICATION 


EFFECTIVENESS 


BMP         OWNERSHIP  12  3  4  5 

tttmmmmt  uutmtn     tttttt  unit  turn  mm  stint 

ROADS  - 
MAINTENANCE 


i  2  3  J  5 

jiJin  nun  iikii  nun  unit 


3.  AVOID  FOE   NON-IND.PRIV 
SLOPE  CUTS  PLUM  CREEK 
STATE  LANDS 
USFS 

mm  turn  tttttt  nun  tttttt 

TOTAL  0  0  0         32  0 


0 

o 

0 

0 

o 

0 

o 

0 

13 

o 

o 

0 

0 

7 

o 

.) 

o 

o 

16 

a 

0           0           0          0  0 

0           0           0         13  0 

0           0           0           3  0 

0           0           0         16  0 

mm  nun  mm  tttttt  tttttt 

0           0           0         32  0 


4.  DRAINAGE  FOP 

CLOSED  ROADS  NON-IND.PRIV 

0 

0 

0 

o 

0 

PLUM  CREEK 

o 

0 

4 

t 

o 

«   STATE  LANDS 

I) 

0 

0 

4 

0 

USFS 

o 

o 

1 

7 

0 

rOTAL 


nun  mm  mm  mm  tttttt 

0     0     5    15     0 


0  0  0  0  0 
0  0  5  3  0 
0  0  0  4  0 
0  0  17  0 
tttttt  tttttt  tttttt  tttttt  tttttt 
0     0     6    14     0 


5.  RESTRICTED 
WET  -  PERIOD 
USE 


NON-IND.PRIV 

0 

0 

1 

t 

L 

0 

0 

0 

1 

2     0 

PLUM  CREEK 

0 

o 

L 

x  J 

o 

0 

0 

L 

13     0 

STATE  LANDS 

0 

o 

o 

4 

o 

0 

o 

0 

4     0 

USFS 

o 

0 

(I 

17 

o 

0 

o 

1 

Is     0 

tttttt 

tttttt 

tttttt 

tttttt 

tttttt 

mm 

tttttt 

tttttt 

tttttt  tttttt 

TOTAL 

0 

0 

3 

36 

1) 

0 

0 

4 

35    0 

TIMBER  HARVEST 

I.  ADEQUATE  SMZ  NON-IND.PRIV 
PLUM  CREEK 
STATE  LANDS 
USFS 

i 

L 

o 

ft 

0 

0 

1 

0 
3 

9 
i, 

3 
0 
3 

0 

? 

3 

16 

t 

6 
3 
0 

TOTAL 


tttttt  mm  ttttu  tttttt  tttttt 


3 


28 


10 


10     2    2    0 

1  0  1  17  0 
0  0  0  6  0 
0     0     6    16     0 

tttttt  tttttt  tttttt  tttttt  tttttt 

2  0     9    41     0 


2.  STREAMS  FREE 

OR  DEBRIS   NON-IND.PRIV 
PLUM  CREEK 
STATE  LANDS 
USFS 

TOTAL 


2  0           12  0 

0  1           1          16  0 

0  0           0           6  0 

0  0           1          21  0 

tttttt  ttttu  tttttt  tttttt  tttttt 

2  1           3          45  0 


1 

1 

o 

0 

o 

o 

1 

17 

0 

0 

o 

0 

6 

o 

o 

o 

n 

L 

20 

o 

mm  tttttt  tttttt  tttttt  tttttt 

1     1     3    46     0 
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Table     6.      (continued) 


APPLICATION  EFFECTIVENESS 

BMP         OWNERSHIP  12*45  1  2  7  ^  = 

miiiiiiiiim  uutututt    mm  tttttt  inm  mm  iiiiii  mm  mm  mm  mm  mm 


14 


tutu  tttttt  tttttt  mm  mm  mm  mm  nun  mm  mm 

TOTAL  0  1  5         23  0  0  12         31  0 


3.  AVOID  EQUIP. 

IN  HET  AREAS  NGN-IND.FRIV 

0 

o 

1 

PLUH  CREEK 

o 

o 

1 

STATE  LANDS 

0 

o 

1 

USFS 

o 

1 

L 

4.  fllNiriIZE 

SOD  DISTURB  NON-IND.PRIV 

0 

n 

o 

5 

PLUM  CREEK 

0 

0 

V 

19 

STATE  LANDS 

0 

o 

0 

4 

•   USPS 

o 

o 

0 

20 

2  ■}           0           0  &           0 

2  0           0           0  22           3 

tttttt  tttttt  mm  mm  tttttt  mm  tttttt  tutu  tttttt  tutu 

TOTAL                         0           0           0          47           5  0           0:  51           9 

5,  SKID  TRAIL 

LOCATION  ADEQ  NON-IND.PRIV  1112  0 

PLUM  CREEK  0  1  0  IS  1 

STATE  LANDS  0  0  0  6  0 

USFS  0  0  3         U  0 

tttttt  tttttt  tttttt  tttttt  tttttt 

TOTAL  1  2  4  40  1 


o 

I 

l 

3     0 

0 

t 

o 

15     0 

,•} 

0 

o 

6     0 

•:• 

o 

2 

IS     0 

itttli 

tttttt 

mm 

tttttt  tttttt 

0 

-1 
L 

7 

43     0 

6.  SKID. MINIMIZE 

SOIL  COMPACT. 

NON-IND.PRIV 

0 

o 

o 

4 

0 

PLUM  CREEK 

0 

0 

,;) 

13 

1 

STATE  LANDS 

o 

o 

o 

6 

o 

USFS 

o 

0 

o 

21 

o 

tttttt 

tttttt 

tttttt 

Utttt 

utttt 

TOTAL 

o 

0 

o 

4? 

1 

7.  WATER  3ARS 

INSTALLED  ... 

NON-IND.PRIV 

o 

o 

1 

i 

1 

o 

PLUM  CREEK 

A 

1 

2 

11 

o 

STATE  LANDS 

o 

o 

0 

n 

0 

USFS 

0 

("i 

7 

12 

0 

utttt 

utttt 

utttt 

tttttt 

tttttt 

TOTAL 

o 

6 

26 

0 

0 

o 

o 

4 

0 

0 

0 

0 

19 

Q 

0 

0 

0 

5 

o 

o 

o 

0 

21 

o 

tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     0    50     0 


0     0 

1 

1     0 

0     2 

1 

11     0 

0     0 

0 

2     0 

0     0 

1 

14     0 

tuu  tttttt 

tttttt 

tttttt  tttttt 

0     2 

J 

29     0 
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Table      6.      (continued) 


APPLICATION 


EFFECTIVENESS 


BMP         OWNERSHIP 

1 

L 

3 

4 

5 

mmttmtmt  mtmim 

tutu 

mtit 

ttutt 

mm 

tutu 

TIMBER  HARVEST 

3.  LANDING  SIZE 

AND  LOCATION  NON-IND.FRIV 

o 

o 

l 

4 

0 

PLUfl  CREEK 

0 

l 

l 

17 

rt 

STATE  LANDS 

o 

o 

o 

i 

!.' 

USFS 

o 

o 

o 

22 

o 

TOTAL 


mm  mm  mm  mm  mm 

0  1  2         4?  0 


1  2  3  4  5 

mitt  ttittt  nun  mm  mm 


0          0  14          0 

0           0  2         17           0 

I)               0  1                i               0 

0           0  0         22          0 

mm  tttltt  nun  mm  mm 


0 


y 


4? 


0 


LOGGING 

SYSTEM 

NOH-IND.PRIV 

0 

0 

o 

5 

0 

o 

0 

0 

5 

0 

PLUH  CREEK 

o 

u 

i 

17 

1 

0 

o 

1 

18 

0 

STATE  LANDS 

o 

0 

o 

6 

0 

0 

0 

o 

6 

0 

USFS 

0 

0 

1 

21 

o 

0 

o 

0 

22 

1) 

TOTAL 


mm  mm  ana  mm  mttt 

0  0  2         4?  1 


mm  mttt  tttttt  tttttt  mm 

0  0  1         51  0 


10.  WASTE 


DISPOSAL 

NON-IND.PRIV 

0 

o 

1 

4 

0 

0 

o 

0 

5 

0 

PLUM  CREEK 

r. 

o 

o 

1? 

0 

o 

o 

0 

19 

o 

STATE  LANDS 

0 

o 

l) 

a 

o 

0 

o 

o 

6 

n 

USFS 

0 

0 

o 

Li. 

o 

o 

o 

o 

0 

tttttt 

tttttt 

tltttt 

mttt 

mm 

tttttt 

mttt 

tttttt 

tttttt 

tttttt 

TOTAL 

o 

o 

( 

51 

o 

o 

0 

0 

0 

.  SEASON  OF 

USE 

NON-IND.PRIV 

o 

o 

1 

i 

L 

0 

0 

0 

1 

L 

0 

PLUM  CREEK 

o 

o 

0 

18 

o 

0 

o 

0 

18 

0 

STATE  LANDS 

o 

0 

0 

6 

0 

0 

0 

0 

6 

0 

USFS 

0 

0 

0 

20 

0 

0 

o 

0 

20 

o 

TOTAL 


tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     1    46     0 


tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     1    46     0 


SLASH  TREATMNT 
SITE  PREP. 


1.  BRUSH  FLADES  NON-IND.PRIV 


5N  DOZERS 


PLUM  CREEK 
STATE  LANDS 
USFS 

TOTAL 


0 

0 

0 

16 

0 

o 

o 

0 

4 

0 

o 

0 

0 

12 

o 

tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     '"*    33     0 


0 

0 

0 

1 

0 

0 

0 

0 

16 

0 

0 

0 

0 

4 

o 

0 

1) 

0 

12 

o 

tttttt  tttttt  tttttt  tttttt  tttttt 

0     0     0    33    0 
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Table   6.      (continued) 


APPLICATION  EFFECTIVENESS 


6f!P         QHNERSHIF  12  3  4 


1  i.  0 


UUUUUUUS  miiiiiiiii    tttitt  una  mm  mm  mm  idiii  mm  mm  mm  mm 

2.  SCARIFICATION 

NON-iND.PRIV 
:_<JM  CREEK 
3 TATE  LANDS 
USPS 

mm  mm  tmtt  mm  mm  mm  tutu  tinu  nun  mm 

TOTAL  0  0  2  41  0  0  0  2         *1  0 


0 

0 

1 

« 

0 

y 

) 

0 

16 

o 

0 

0 

o 

6 

0 

o 

o 

1 

17 

0 

c 

o 

1 

i 

o 

0 

0 

0 

16 

5 

.) 

0 

ft 
V 

rt 

g 

0 

o 

1 

1  "7 

o 

'.  OPERATION 

NINIHIZE  : 

SOIL 

NON-IND.PRIV 

0 

0 

0 

3 

0 

0 

0 

0 

3 

0 

compact/d: 

[ST. 

PLUM  CREEK 

o 

0 

c 

10 

0 

0 

0 

3 

12 

0 

STATE  LANDS 

i") 

0 

1 

7 

0 

0 

0 

0 

4 

o 

liSFS 

o 

0 

i 

13 

1 

0 

0 

n 

c 

13 

0 

mm 

mm 

mm 

mm 

turn 

turn 

mm 

turn 

tutu 

nmt 

TOTAL 

o 

0 

7 
/ 

29 

1 

0 

0 

5 

32 

0 

4.  DOZERS  ON 

SUITABLE 

NON-IND.PRIV 

0 

G 

o 

4 

0 

0 

0 

0 

4 

0 

SLOPES 

PLU1  CREEK 

o 

g 

•  0 

16 

0 

0 

0 

0 

16 

0 

STATE  LANDS 

0 

0 

0 

6 

0 

0 

0 

0 

6 

0 

JSFS 

y 

0 

o 

16 

0 

0 

o 

o 

16 

0 

mm  urn*  imn  mm  mm  mm  turn  mm  tutu  mttt 

"TAL                         0           0           0         42           0  0           0           0  42           0 

:.  iK 

PROTECTION       NON-IND.PRIV            10           0           3           0  10           0  3           0 

PLUM  CREEK                0           2           2          13           0  0           0           2  15           0 

STATE  LANDS               0           0           0     .       h           0  0           0           0  6           0 

U5F3                          0           1           6          13           1  0           1           1  19           0 

mm  mm  mm  turn  mitt  mm  mtn  mm  tutu  utm 

TOTAL                         1           3           3          35           1  1            1           3  43           0 


ri'TALS 


.S         NON-IND.PRIV 

11 

7 

20 

65           1 

PLUH  CREEK 

9 

16 

48 

473         12 

STATE  -ANDS 

0 

Q 

jj 

164           6 

U3F3 

0 

7 

^7 

558           5 

mm 

mm 

mm 

mm  mm 

3RAND  TOTAL 

1  ; 

26 

106 

1.260         24 

4 

7 

La 

71 

0 

100 

1 

10 

J  C 

49; 

o 

5*9 

0 

0 

6 

169 

o 

175 

0 

5 

30 

:67 

1 

603 

mm  mm  mm  tutu  tutu  tuuu 

.427  5         22         M    1.300  1     1.427 


QF  TOTAL  5.3        1.3        7.4      S3. 3        1.7     100.0        0.3        1.5        s.9      91.1        0.1     100.0 
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Method  2  -  Analysis  of  Impacts  by  "Percentage  of  Practices 
Audited"  on  Each  Ownership 

This  approach  demonstrates  each  individual  ownership's 
share  of  the  1427  basin-wide  practices  audited  and  how  each 
performed  in  controlling  impacts  when  compared  only  to  the 
practices  they  conducted.   Figures  1  and  2  show  the  results 
of  this  analysis  for  both  practice  application  and 
effectiveness,  for  each  ownership.   Further  explanation  on 
the  meanings  of  these  values  is  provided  in  the 
Methodologies  chapter,  in  the  section  "Rating  BMPs." 

When  short-  and  long-term  impacts  are  summed  and 

compared  to  "total  practices  audited"  for  each  ownership, 

percentages  of  practices  contributing  to  impacts  are 

calculated.   Figure  3  focuses  specifically  on  this  analysis 

and  follows  the  tabulated  numbers  presented  in  Table  7. 

Table  7.   Percentage  of  practices  with  impacts. 

PRACTICES  PRACTICES   CONTRIBUTING 

OWNERSHIP           AUDITED  W/ IMPACTS    TO  IMPACTS 

Non-Indust.  Private      100  29           29.0 

Plum  Creek  Timber  Co.    549  56           10.2 

Dept.  of  State  Lands      175  6            3.4 

Flathead  Nat.  Forest      603  35            5.8 

Totals          1427  126             8.8 

Both  of  the  preceding  methodologies  utilizing  "total 

practices  audited"  are  flawed  in  their  use  as  an  accurate 

indication  of  the  performance  and  success  of  controlling 

sediment  production.   Within  "Total  Practices  Audited"  are  a 

number  of  practices,  approximately  13%  of  the  total  1427, 

that  are  related  to  items  which  either  do  not  directly 
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%  OP  PRACTICES  RATED  POR  APPLICATION 


NON-INDUST.PRIVATE  PLUM  CREEK 


USTS 


STATE  LANDS 


Rating  Values 
2      0   3       ■ 


FIGURE  1 


%  OF  PRACTICES  RATED  FOR  EFFECTIVENESS 


NON-iNDUST  PRIVATE 


PLUM  CREEK 


USFS 


STATE  LANDS 


FIGURE  2 


Rating  Values 
2 
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%  OF  PRACTICES  CONTRIBUTING  IMPACTS 


NON-INDUST.PKIV      PLUM  CREEK 


STATE  LANDS 


%  OF  PRACTICES 


FIGURE  3 


influence  the  production  of  sediment  in  the  stream  channel 
or  are  too  difficult  or  inaccurate  to  attempt  to  assess  in 
the  post  harvest  time  period.   Those  five  practices  are: 

1.  Permits  obtained  for  stream  crossings. 

2.  Timely  installation  of  drainage  features  on  roads. 

3.  Halting  road  construction  during  wet  periods. 

4.  Erosion  control  measures  are  current  with  road 
construction. 

5.  Road  use  restricted  during  wet  periods 
Practices  2  through  5  have  a  potential  to  directly 

impact  sedimentation  but  inspection  several  years  following 
the  activity  does  not  give  an  accurate  picture  of  the 
impacts.   Adding  to  this  potential  problem  is  the  fact  that 
not  all  of  the  39  practices  assessed  in  the  field  carried 
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the  same  weight  in  potential  sediment  production.   This 
point  was  widely  recognized  by  audit  team  members. 

A  practice  such  as  "waste  disposal"  (which  scored  as 
adequately  applied  and  effective  on  all  52  sales)  for  which 
an  impact  could  only  happen  if  equipment  oil  changes 
occurred  next  to  a  channel,  is  really  not  as  important  as  a 
practice  such  as  "adequate  SMZ  maintained"  which  could 
potentially  produce  considerably  more  damage.   They  are, 
however,  weighted  the  same  when  the  numerical  analysis  is 
conducted. 

Relying  on  the  numbers  generated  using  the  "total 
practices"  approach  as  in  the  two  examples  above,  can  be  an 
inaccurate  depiction  of  the  level  of  protection  occurring  in 
the  watershed.   Practices  which  have  a  greater  potential  for 
impact  should  carry  more  weight  in  the  final  analysis. 
Method  3  -  Analysis  of  Ratings  on  a  Timber  Sale  Basis 

A  different  approach  is  to  let  each  timber  sale  stand 
on  its  own  merits  for  representing  the  level  of  practice 
application  and  protection  provided  to  a  stream.   Ratings 
for  BMP  effectiveness  are  summarized  for  all  minor  temporary 
detrimental  impacts  (see  Table  8)  and  for  all  the  major 
impacts,  both  short-  and  long-term  (see  Table  9) .   Only  the 
sales  in  which  impacts  were  observed  are  listed.   In  the 
tables  the  BMP  categories  responsible  for  the  observed 
impacts  are  noted  along  with  the  number  of  practices. 
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Figures  4  and  5  graphically  summarize  the  data  in  the 

tables. 

Table  8.   Timber  sales  with  at  least  one  minor  impact. 


OWNER 


SALE  NAME 


#  IMPACTS   CATEGORY 


NON- INDUS. 

PRIVATE     1 . 

BROWNS  MEADOW 

2 

TIMBER  HARVEST 

2. 

SALMON  PRAIRIE 

2 

ROAD  PLANNING 

1 

ROAD  DRAINAGE 

3. 

PABLO 

2 

ROAD  DRAINAGE 

1 

ROAD  MAINTENANCE 

2 

TIMBER  HARVEST 

4. 

STAR  MEADOWS 

2 

ROAD  PLANNING 

1 

ROAD  CONSTRUCTION 

1 

ROAD  MAINTENANCE 

3 

TIMBER  HARVEST 

1 

SITE  PREP. 

#  OF  SALES:   4 

18 

PERCENT 

80.0%       (    4 

out  of  5  sales) 

PLUM  CREEK 

1. 

SOUTH  COLD 

1 

ROAD  MAINTENANCE 

2. 

SQUAW  CAMP 

2 

ROAD  DRAINAGE 

BROOK  EAST 


4.  JIM  CORNICE 

5.  LION  FLAT 

6.  G-BRANCH  II 

7.  OWL  ONE  QTR. 


8.  LOWER  TAMARACK 

9.  BROWN'S  MEADOW  LP 

10.  DEAD  ON  IT'S  FEET 

11.  BERNARD  FLAT 

12.  AB  LODGEPOLE 

13.  WEST  JIM 


1  ROAD  CONSTRUCTION 
3  ROAD  MAINTENANCE 

2  TIMBER  HARVEST 

1  SLASH  TREAT/ SITE  PREP 

2  ROAD  DRAINAGE 

1  ROAD  CONSTRUCTION 

3  ROAD  MAINTENANCE 

2  ROAD  DRAINAGE 

2  ROAD  CONSTRUCTION 

1  ROAD  DRAINAGE 

1  SLASH  TREAT/ SITE  PREP 

1  ROAD  PLANNING 

2  ROAD  DRAINAGE 

1  ROAD  MAINTENANCE 

3  TIMBER  HARVEST 

2  ROAD  DRAINAGE 

1  ROAD  MAINTENANCE 

1  SLASH  TREAT/ SITE  PREP 

1  ROAD  MAINTENANCE 

1  SLASH  TREAT/ SITE  PREP 

1  SLASH  TREAT/ SITE  PREP 

2  ROAD  MAINTENANCE 

1  ROAD  DRAINAGE 

2  ROAD  CONSTRUCTION 
1  ROAD  MAINTENANCE 
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Table  8  (continued) 


OF  SALES:   13 

PERCENT     68.4% 


45 


TIMBER  HARVEST 


(13  out  of  19  sales) 


USPS 


1.  NINKO  MILLER 


2.  LOGAN  FALLS 

3.  BILL  CREEK 

4.  DUNN  TEPEE 

5.  COAL  RIDGE 


NAPA  GOAT 


7.  MIDDLE  FORK  LP 

8.  TERRACE  HILL 

9.  SANDERS  HAND 

10.  MISSION  BUTTE 

11.  ELK  COLD 

#  OF  SALES:   11 
PERCENT   50.0% 


3 

1 
2 

1 
1 
1 
1 
1 
2 
4 
2 
1 
1 
1 
1 
1 
1 
2 
2 
1 


ROAD  DRAINAGE 
ROAD  CONSTRUCTION 
ROAD  MAINTENANCE 
ROAD  PLANNING 
TIMBER  HARVEST 
TIMBER  HARVEST 
ROAD  PLANNING 
ROAD  DRAINAGE 
ROAD  CONSTRUCTION 
TIMBER  HARVEST 
SLASH  TREAT/ SITE  PREP 
ROAD  PLANNING 
ROAD  MAINTENANCE 
TIMBER  HARVEST 
ROAD  DRAINAGE 
TIMBER  HARVEST 
ROAD  MAINTENANCE 
SLASH  TREAT/ SITE  PREP 
TIMBER  HARVEST 
TIMBER  HARVEST 


30 


(11  out  of  22  sales) 


STATE  LANDS 


1.  LOWER  CILLY 

2.  SOUTH  COAL  CREEK 


3.  SWAN  RIVER 

6 
#  OF  SALES:   3 
PERCENT   50.0% 


1 
1 
1 
1 
1 
1 


TIMBER  HARVEST 
ROAD  PLANNING 
ROAD  DRAINAGE 
ROAD  CONSTRUCTION 
ROAD  MAINTENANCE 
ROAD  DRAINAGE 


(3  out  of  6  sales) 


ALL  OWNERSHIPS 

PERCENT   59.6% 


(31  out  of  52  sales) 
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Table  9.   Timber  sales  with  at  least  one  major  impact 

#  OF  CONTRIBUTING  PRACTICES 

RELATED  TO  EACH  SALE: 
SHORT  TERM  LONG  TERM 

IMPACTS  IMPACTS 


OWNER  /  SALE  NAME 

OTY 

CATEGORY        OTY 

CATEGORY 

NON-IND. PRIVATE 

1.  BROWNS  MEADOW 

1 

TIMBER  HARVEST 

— 

— 

2.  STAR  MEADOWS 

1 

ROAD  PLANNING 

— 

— 

2 

ROAD  DRAINAGE 

- 

- 

2 

ROAD  CONSTRUCT 

- 

- 

1 

TIMBER  HARV. 

2 

TIMBER  HARV. 

1 

ROAD  MAINTEN. 

1 

SLASH/SITE  P. 

#  OF  SALES:   2 

8 

3 

PERCENT   40  % 

(  2  out  of  5  sales 

) 

PLUM  CREEK 

1.  SQUAW  CAMP 

1 

ROAD  PLANNING 

- 

- 

1 

ROAD  DRAINAGE 

- 

- 

2 

TIMBER  HARV. 

1 

TIMBER  HARV. 

2.  JIM  CORNICE 

1 

TIMBER  HARV. 

- 

- 

3.  WEST  JIM 

2 

ROAD  PLANNING 

- 

- 

1 

ROAD  DRAINAGE 

- 

- 

1 

RD.  CONSTRUCT 

- 

- 

1 

ROAD  MAINT. 

- 

- 

#  OF  SALES:   3 

10 

1 

PERCENT   15.8% 

( 

3  out  of  19  sales 

) 

USPS 

1.  LOGAN  FALLS 

1 

ROAD  MAINT. 

- 

- 

2.  BILL  CREEK 

1 

TIMBER  HARV. 

- 

- 

1 

SLASH/SITE  P. 

- 

- 

3.  NAPA  GOAT 

2 

ROAD  DRAINAGE 

— 

#  OF 

SALES :   3       5 

0 

PERCENT    13.6% 

( 

3  out  of  22  sales) 

DEPT  OF  STATE  LANDS 
#  OF  SALES:   0 
PERCENT     0.0% 

ALL  OWNERSHIPS 

PERCENT    15.4% 


(0  out  of  6  sales) 


(8  out  of  52  sales) 
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40. OX 


30. OX  - 


20  OX  - 


10.0X 


0  OX 


X  OF  SALES  W/AT  LEAST  1  MAJOR  IMPACT 


SMALL  PRIVATE  PLUM  CREEK  USFS  STATE  LANDS     ALL  OWNERS 


PERCENTAGE  W/IMPACTS 


FIGURE  * 


X  OF  SALES  W/AT  LEAST  1  MINOR  IMPACT 


00.  ox 

• 

80. OX 

60. OX 

400X 











20.0X 

-- J 





SMALL  PRIVATE  PLUM  CREEK  USFS  STATE  LANDS  ALL 


PERCENTAGE  W/IMPACTS 


FIGURE  5 
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COMPARISON  OF  PERFORMANCE  BETWEEN  THE  FLATHEAD 
BASIN  AND  THE  STATE  OF  MONTANA 


The  Montana  Environmental  Quality  Council  (EQC) 
utilized  the  following  approaches  in  analyzing  the  results 
of  the  1988  statewide  BMP  assessment: 

1.  %  of  Total  Practices  in  Each  Rating  Category 

2.  Number  of  Sales  with  Various  Levels  of  Departure 
and  Impacts 

a.  #  with  at  least  1  major  practice  departure 

b.  #  with  at  least  1  practice  rating  as  a 
major  impact 

3.  Number  of  Significant  Problems  Per  Timber  Sale 
Each  of  these  methods  will  be  examined  and  comparisons  made 
in  to  order  to  establish  whether  there  are  differences  in 
the  results  of  the  two  assessments. 

Table  10  shows  a  comparison  between  the  "%  of  Total 
Practices  in  Each  Rating  Category."    The  grouping  of  data 
from  5  classes  to  3  classes  was  done  to  accommodate  the  use 
of  a  statistical  method  to  test  the  probability  of  no 
significant  difference  between  the  results  of  these  studies. 
Graphical  comparisons,  found  in  Figures  6  and  7,  show  how 
the  3  classes  compare  between  the  Flathead  Basin  and  EQC 
audits. 

The  Chi-Squared  (X  )  Test  was  conducted,  and  is  well 
suited  for  testing  hypotheses  about  independent  samples  of 
ordinal  data.   The  calculations  are  illustrated  in  Table  11. 
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The  result  of  this  analysis  is  that  we  cannot  reject  the 
hypothesis  that  there  is  "no  significant  difference"  between 
the  results  of  these  two  studies.   X  values  as  large  as  we 
observed  indicates  that  the  differences  are  probably 
significant  and  not  due  to  chance. 


Table  10.   Comparison  of  the  1989  Flathead  Basin  BMP 
assessments  with  the  1988  statewide  BMP  assessments. 


APPLICATION  OF  BMPS 


BMP 

MAJOR 

MINOR 

ADEQUATE 

OR 

STUDY 

DEPARTURES 

DEPARTURES 

EXCEEDS  : 

BMPs 

(ratinq  1-2) 

(ratinq  3) 

(ratinq  4- 

5) 

TOTAL 

EQC  STATEWIDE  (38)  * 

#  OF  PRACTICES 

45 

126 

754 

925 

%  OF  PRACTICES 

5% 

14% 

81% 

100% 

FLATHEAD  BASIN  (52) * 

#  OF  PRACTICES 

37 

106 

1286 

1429 

%  OF  PRACTICES 

3% 

7% 

90% 

100% 

EFFECTIVENESS  OF 

BMPS 

BMP 

MAJOR 

MINOR 

ADEQUATE 

OR 

STUDY 

IMPACTS 

IMPACTS 

BETTER 

(ratinq  1-2) 

(ratinq  3) 

(ratinq  4- 

-5) 

TOTAL 

EQC  STATEWIDE: 

#  OF  PRACTICES 

31 

117 

777 

925 

%  OF  PRACTICES 

4% 

13% 

83% 

100% 

FLATHEAD  BASIN: 

#  OF  PRACTICES 

27 

99 

1303 

1429 

%  OF  PRACTICES 

2% 

7% 

91% 

100% 

1  -  Represents  the  number  of  sites  audited  in  each  study. 
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COMPARISON  OF  BMP  APPLICATION 


X  OF  TOTAL  PRACTICES 


FIGURE  8 


MAJOR  DEPARTURES  MINOR  DEPARTURES  ADEQUATE  OR  EXCEEDS 

CATEGORIES  OF  BMP  APPLICATION 

HI    EQC  STUDY        mi    FLATHEAD  STUDY 


COMPARISON  OF  BMP  EFFECTIVENESS 


Z  OF  TOTAL  PRACTICES 


nouRE  7 


MAJOR  IMPACTS  MINOR  IMPACTS  ADEQ.  PROTECTION 

CATEGORIES  OF  BMP  EFFECTIVENESS 

■I    EQC  STUDY        NWVM    PLATHEAD  STUDY 
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Table  11.   The  X2  Test  For  Significant  Difference  Between 
the  EQC  and  Flathead  Basin  BMP  Assessment  Results 

HYPOTHESIS:  THERE  IS  NO  SIGNIFICANT  DIFFERENCE  BETWEEN  THESE 
RESULTS . 

EQUATION:    X2  =  SUM  ALL  (OBSERVATIONS  -  EXPECTED  VALUE 1 2 

EXPECTED  VALUE 

OBSERVATIONS     =  FLATHEAD  BASIN  RESULTS 

EXPECTED  VALUES  =  P  x  n 

EQC'S  PROPORTION  OF  OBSERVED  RATINGS 
(P) ,  FOR  EACH  OF  3  CLASSES,  MULTIPLIED 
BY  THE  SAMPLE  SIZE  (n) . 

n  =  1427,  SAMPLE  SIZE  OF  PRACTICES  IN  THE  FLATHEAD  STUDY 
d.f.  =  2 


BMP  APPLICATION:   RATING  CLASSES 

MAJOR        MINOR      ADEQUATE  OR 
DEPARTURE    DEPARTURE    EXCEEDS  BMP 
PRACTICES      (rating  1-2)  (rating  3)  (rating  4-5)   TOTAL 

OBSERVED   (O)      37  106  1284        1427 

EXPECTED  .05  X  1427   .14  X  1427    .81  X  1427 

(E)  =  P  X  n       =71         =  200        =  1156 

CHI  -  SQUARE   CALCULATING   TABLE  : 

O  -  E  =  D  34  94  128 

D2  1156  8836        16,384 

X2  =  SUM  ALL  D2/E:  16.28    +     44.18    +     14.17 
74.63 

USING  OUR  X2  VALUE  OF  74.63,  AND  REFERRING  TO  A  TABLE  OF  X2 
PROBABILITY  VALUES  USING  2  "DEGREES  OF  FREEDOM"  YIELDS: 

THE  PROBABILITY  OF  "NO  SIGNIFICANT  DIFFERENCE"   <   0.1  % 

THIS  INDICATES  THE  DIFFERENCES  OBSERVED  IN  THE  APPLICATION 
OF  BMPs  BETWEEN  THE  TWO  STUDIES  IS  ALMOST  CERTAINLY 
SIGNIFICANT  AND  NOT  DUE  TO  CHANCE. 
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Table  11  (continued) 


BMP  EFFECTIVENESS:  RATING  CLASSES 

MAJOR         MINOR     ADEQUATE  OR 
IMPACTS       IMPACTS     BETTER 
PRACTICES       (rating  1-2)   ( rating  3)  (rating  4-5)   TOTAL 

OBSERVED   (0)  27  99  1300       1427 

EXPECTED   (E)      .035  X  1427   .13  X  1427   .835  X  1427 

=50  =  186        =  1192 

CHI-SQUARED  CALCULATING  TABLE: 

0  -  E  =  D              23  87           108 

D2                     529  7569         11664 

X2  =  SUM  ALL  D2/E:  10.58     +  40.69  +     9.78    =   61.06 

USING  THE  X2  VALUE  OF  61.06,  WITH  2  DEGREES  OF  FREEDOM,  IN 
THE  X2  TABLE  YIELDS: 

A  PROBABILITY  OP  "NO  SIGNIFICANT  DIFFERENCE"   <   0.1  % 

THIS  INDICATES  THE  DIFFERENCES  OBSERVED  IN  THE  EFFECTIVENESS 
OF  BMPs  IN  MITIGATING  RESOURCE  IMPACTS,  BETWEEN  THE  TWO 
STUDIES,  IS  ALMOST  CERTAINLY  SIGNIFICANT  AND  NOT  DUE  TO 
CHANCE. 


Another  approach  to  comparing  the  results  of  these  two 
studies  looks  at  the  percentage  of  timber  sales  displaying 
departures  from  BMPs  and  impacts.    For  each  study,  a 
combined  percentage  of  timber  sales  with  "at  least  one  major 
practice  departure"  for  application  and  "at  least  one  major 
impact"  for  effectiveness  is  calculated.     The  Flathead 
Basin  results  for  the  two  values  were  19.2  %  and  15.4  %, 
respectively.   The  statewide  results  were  52.6  %  and  42.1%, 
respectively.   These  values  are  shown  in  Figure  8. 
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PERCENTAGES  OF  TIMBER  SALES 


FLATHEAD  BASIN 


MONTANA  (EQC) 


W/  >-  1  MAJ    DEPART 


W/  >-  1  MAJ.  IMPACT 


FIGURE  8 


A  third  comparison  looks  at  the  number  of  actual 
impacts  per  sale  as  an  indicator  of  "significant  problems 
per  sale."   This  is  achieved  by  summing  the  short-  and  long- 
term  major  impacts  recorded  for  each  study  and  dividing  by 
the  number  of  sales  audited.   The  same  calculation  is  then 
done  for  the  minor  impacts.   The  results  for  the  Flathead 
Basin  study  are  .52  for  major  impacts  and  1.9  for  minor 
impacts  compared  to  .82  and  3.1,  respectively,  for  the 
State.   Figure  9  shows  these  comparisons. 

This  analysis  indicates  that  in  the  Flathead  Basin, 
nearly  1  major  impact  is  occurring  for  every  2  timber  sales 
conducted.   Similarly,  almost  2  minor  impacts  are  occurring 
on  every  timber  sale  harvested. 
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#'S  OP  SIGNIFICANT  PROBLEMS  /  SALE 


FLATHEAD  BASIN 


MONTANA  (EQC) 


MAJOR  IMPACTS/SALE 


MINOR  IMPACTS/SALE 


FIGURE  9 


ANALYSIS  OF  IMPACTS  BY  BMP  CATEGORIES 

This  analysis  is  intended  to  demonstrate  categories  of 
BMP  implementation  that  are  in  need  of  improvement.   Table 
12  summarizes  major  and  minor  impacts  by  respective  BMP 
categories. 

The  Timber  Harvest  and  Road  Drainage  categories  had  the 
greatest  number  of  impacts  with  Road  Maintenance  following 
closely  behind.   Figure  10  shows  how  each  ownership 
performed  in  each  BMP  category  when  minor  and  major  impacts 
were  summed  and  taken  as  a  percentage  of  the  total  impacts. 
Streamside  Management  Zones 

Adeguate  Maintenance  of  the  Streamside  Management  Zone 
(SMZ)  had  the  greatest  number  of  departures  and  impacts. 
Twenty-seven  departures  out  of  the  total  of  143  (19%)  were 
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Table  12.   Summary  of  BMP  categories  contributing  to  soil 
and  water  impacts. 


MINOR  IMPACTS:  #  OF  PRACTICES  CONTRIBUTING  TO  IMPACTS 


TOTAL 


18   100 


45   100 


30   100 


100 


NON- 

-IND, 

i 

PLUM 

STATE 

TOTAL 

PRIVATE 
OTY   % 

* 

CREEK 
OTY    % 

* 

USFS 

* 

LANDS 
OTY   % 

* 

PCT. 

BMP  CATEGORY 

QTY 

% 

%  * 

ROAD  PLANNING 

4 

22 

* 

1 

2 

* 

1 

3 

* 

1 

17 

* 

7 

ROAD  DRAINAGE 

3 

17 

* 

12 

27 

* 

7 

23 

* 

2 

33 

* 

24 

ROAD  CONSTRUCT. 

1 

5 

* 

6 

13 

* 

3 

10 

* 

1 

17 

* 

11 

ROAD  MAINTEN. 

2 

11 

* 

14 

31 

* 

4 

13 

* 

1 

17 

* 

21 

TIMBER  HARVEST 

7 

39 

* 

7 

16 

* 

11 

37 

* 

1 

17 

* 

26 

SLASH  TREATMENT  1 

5 

* 

5 

11 

* 

4 

13 

* 

0 

17 

* 

10 

100 


MAJOR  IMPACTS:    #  OF  PRACTICES  CONTRIBUTING  TO  IMPACTS 


BMP  CATEGORY 

NON-IND. 
PRIVATE 

PLUM 
CREEK 

USFS 

STATE 
LANDS 

TOTAL 

PCT. 

ROAD  PLANNING 
ROAD  DRAINAGE 
ROAD  CONSTRUCTl 
ROAD  MAINTENANC 
TIMBER  HARVEST 
SLASH  TREATMEN1 

:on 
:e 

1 

1 

2 
2 

1 
4 
1 

3 
2 

1 
1 
4 
0 

0 
2 

0 

1 
1 
1 

0 
0 
0 
0 
0 
0 

4 
6 
3 
3 
9 
2 

15 
22 
11 
11 
33 
7 

TOTAL 

11 

11 

5 

0 

29 

100% 

COMBINED  TOTAL 
MINOR  &  MAJOR 

29 

56 

35 

6 

126 

TOTAL  #'S  OF 
PRACTICES 

100 

549 

603 

175 

1427 

PCT.  OF  TOTAL 

29% 

10% 

6% 

3% 

9% 
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%  OF  [MPACTS  BY  BMP  CATEGORY 


Road  PLan       Drainage       Construct.        Mainten. 

I    Small  Private      KWWVi    pium  Creek         t:    ^    USTS 
All 


Harvest 


Slash 


State  Lands 


FIGURE  10 

attributable  to  the  practice  of  SMZ  protection,  both  during 
the  harvest  and  during  slash  reduction. 

Rating  the  practice  involved  assessing  the 
effectiveness  of  the  vegetative  buffer  zone  along  a  riparian 
area  and  its  capacity  to  act  as  a  filter  for  sediment,  a 
zone  of  stabilization  for  the  channel,  and  to  provide  shade 
and  woody  debris  recruitment  for  fisheries  habitat. 
Departures  in  this  practice  could  include  damage  to  the 
stream  bank  or  bed  due  to  operation  of  eguipment  across  the 
stream  channel  during  skidding  or  slash  piling,  insufficient 
stand  of  timber  or  vegetative  cover  left  to  provide  shading 
and  temperature  control  and  excessive  soil  disturbance  on 
the  flood  plain  and  adjacent  to  the  channel.   Teams  did  not 
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rate  this  area  as  a  "zone  of  exclusion"  from  the  harvesting 
of  timber,  but  instead  rated  it  on  it's  ability  to  function 
effectively. 

Teams  did  not  want  to  see  equipment  operating  in  such  a 
way  that  it  breached  the  natural  topographic  break  of  an 
incised  channel.   Teams  also  scored  practices  lower  if 
excessive  soil  disturbance  was  observed  at  the  immediate 
edge  of  such  a  slope  break.    Operation  of  equipment  on 
these  slopes  not  only  increases  the  risk  that  displaced  soil 
will  move  under  the  forces  of  gravity  into  the  riparian  area 
or  flood  plain,  it  also  sets  up  a  network  of  ruts  from  the 
treads  or  tracks.   These  provide  increased  efficiency  for 
transport  of  water  and  sediment  downslope,  increasing 
potential  impacts  to  streams.   Instead  of  measuring  out  some 
distance  from  the  middle  of  the  stream  channel  to  establish 
an  SMZ,  it  was  generally  felt  that  simply  staying  up  on 
these  benches  above  natural  slope  breaks  would  significantly 
reduce  the  potential  for  channel  impacts. 

On  deeply  incised  channels,  where  there  is  considerable 
distance  between  the  activity  on  the  slope  and  the  channel, 
equipment  operation  below  the  natural  topographic  slope 
break  was  considered  appropriate.   However,  there  should  be 
sufficient  surface  roughness  with  ground  cover  and  woody 
debris  in  the  designated  SMZ  to  detain  any  soil  which  may  be 
detached  upslope,  and  dissipate  any  energy  associated  with 
surface  runoff.   If  an  adequate  buffer  can  be  established, 
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then  the  limiting  factors  in  these  instances  becomes  soil 
retention  on  the  slope  for  productivity  and  the  safe 
operation  of  equipment. 

Wet  sites  displaying  numerous  stream  channels  or 
shallow  water  tables  were  recognized  as  difficult  situations 
for  maintaining  an  adequate  SMZ.   Any  stand  of  timber  left 
to  provide  an  effective  SMZ  was  recognized  as  susceptible  to 
"blowdown."   In  other  instances,  excessive  soil  disturbance 
by  equipment  can  break  through  the  surface  soil  layer  and 
expose  a  new  channel  for  conveyance  of  water  on-site. 
Operating  on  frozen  soils  is  recognized  as  one  method  of 
addressing  the  latter  problem. 

The  EQC's  BMP  Technical  Committee  produced  a  definition 
for  an  SMZ  that  included  a  minimum  width  of  25  feet  as  a 
management  objective.   Since  the  teams  would  be  rating  the 
adequacy  of  the   streamside  management  zone  in  protecting 
soil  and  water  resources  for  52  timber  sales,  we  had  the 
opportunity  to   answer  2  additional  questions  regarding  the 
width  of  these  zones.   Those  findings  are  summarized  below, 
and  provide  some  insight  into  where  we  have  been  with 
respect  to  the  application  of  a  buffer  zone. 

These  data  are  not  intended  to  assess  compliance  with 
the  recent  "25  foot"  definition  which  has  not  been  in 
existence  long  enough  to  provide  ample  implementation  time. 
Instead,  we  gathered  these  data  to  record  the  state  of 
practice  prior  to  the  recent  definition. 
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1.   Was  a  25  foot,  minimum  width,         #'s  of  Sales 

SMZ  maintained  ?  YES     NO     TOTAL 

NON-INDUS  PRIV.  14        5 

PLUM  CREEK  14        5       19 

STATE  LANDS  6        0        6 

USFS  17        5       22 


TOTAL         38       14       52 
PERCENT      73.1%    26.9%   100.0% 

#  of  sales  displaying  impacts 
associated  with  application  of 

the  25  foot  SMZ  criteria:  3       8       11 

27%      73%      100% 

On  8  of  the  11  timber  sales  (73%) ,  the  SMZ  width  was 
less  than  25  feet  and  impacts  directly  related  to  this 
criteria  were  observed.   On  3  sales,  impacts  were  noted  even 
though  a  25'  buffer  had  been  provided.   In  one  case  the  thin 
stand  of  trees  left  as  an  SMZ  was  damaged  by  wind,  leaving 
behind  an  inadequate  filter  strip.   In  2  other  instances  the 
25'  slope  distance  was  inadequate  (given  the  incised  channel 
and  steep  (80%+)  slope)  to  trap  the  detached  soil.   This 
seems  to  support  the  position  that  a  25  foot  minimum  width 
SMZ  may  be  inadequate  to  provide  sufficient  protection  of 
the  water  resources  in  many  instances. 

It  was  noted  that  leaving  a  25*  wide  stand  of  trees 
may,  in  fact,  increase  the  risk  of  blowdown  on  some  sites. 
Removal  of  the  stand  in  a  well-planned  and  executed 
treatment  could  prevent  the  introduction  of  large  amounts  of 
sediment  into  the  channel  at  a  later  date.   In  some  areas, 
the  removal  of  an  adjacent  timber  stand  clearly  "sets  up" 
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the  riparian  areas  for  a  potential  blowdown  event,  and 
damage  to  the  channel.   More  work  is  needed  to  explore 
treatment  options  designed  specifically  to  minimize  the 
potential  of  increased  sedimentation  from  these  events. 

On  6  sales  the  SMZ  was  less  than  25'  and  yet  no  impacts 
were  documented.   In  these  cases  equipment  operation  in  the 
channel  during  yarding,  slash  piling  in  the  channel  and 
excessive  soil  disturbance  were  noted.   These  were  in  poorly 
defined  ephemeral  and  intermittent  channels.   While  the 
teams  found  the  practices  associated  with  the  SMZ  to  be  a 
departure  from  preferred  practices,  it  was  difficult  to 
determine  if  sediment  resulting  from  the  disturbance  had 
moved  down  the  channel  and/or  off -site.   The  fact  that 
impacts  hadn't  been  recognized  was  viewed  possibly  as  a 
result  of  recent  historically  low  precipitation  and  the 
timing  of  runoff  events,  or  combination  of  events.   However, 
the  lack  of  impacts  was  not  viewed  as  an  appropriate 
justification  for  relaxing  the  SMZ  requirement  in  these 
headwater  drainages,  as  problems  were  viewed  on  other 
similar  sites  where  the  combination  of  events  resulted  in 
dramatically  different  results. 

Using  landtype  maps  and  the  professional  judgement  of 
the  team  members,  each  site  was  characterized  for  erosion 
potential.   Of  the  11  timber  sales  on  which  impacts  related 
to  SMZ  maintenance  were  noted,  3  (27%)  were  on  "low"  risk 
sites  while  8  (73%)  were  on  "moderate"  risk  sites. 
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2.   Is  25  feet  an  adequate  distance 
for  an  SMZ,  given  the  site's 

physical  characteristics  and    #'s  of  Sales 

harvest  prescription  ? 


ion  ? 

YES 

NO 

TOTAL 

NON-INDUS  PRIV. 
PLUM  CREEK 
STATE  LANDS  X 
USFS 

4 
13 
2 

15 

1 
6 
2 

7 

5 
19 
4 
22 

TOTAL 
PERCENT 

34 
68.0% 

16 

32.0% 

50 
100.0% 

1)   Two  of  the  State  Lands  sales  did  not  have  complete 
data. 

The  following  reasons  were  recorded  explaining  why  a  25 

foot  SMZ  width  was  deemed  inadequate  on  3  2%  of  the  timber 

sales. 

1.  Deeply  incised  channels  with  40%  +  adjacent  slopes 
leading  down  to  stream  banks  provided  too  efficient  delivery 
of  sediment  to  the  stream,  with  inadequate  filtering 
capability. 

2.  A  25  foot  distance  would  leave  the  edge  of  the 
buffer  below  a  natural  topographic  break  between  upland  and 
riparian  areas.   Any  operation  of  equipment  below  the 
natural  break  was  viewed  as  too  great  a  risk  for  excessive 
soil  and/or  channel  disturbance.   A  25  *  buffer  was  only 
acceptable  if  cable  yarding  had  been  exercised. 

3.  Either  a  high  water  table,  as  characterized  by  the 
prevalence  of  wet  or  moist  vegetation  types,  or  a  meandering 
stream  channel  were  situations  where  the  teams  preferred  to 
see  wider  buffer  strips  left  intact. 
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4.  Evidence  of  an  areas  susceptibility  to  blowdown. 

5.  High  fishery  values  associated  with  some  streams. 

6.  Fine  textured  soils,  lacking  in  cohesion  with  an 
absence  of  coarse  fragments. 

CONCLUSIONS 

At  the  present  time  there  is  no  proven  link  between  the 
gualitative  results  obtained  from  BMP  audits  and 
quantitative  impacts  which  may  result  in  stream  channels. 
Accelerated  sediment  production  is  still  recognized  as  the 
primary  forest  management-related  water  quality  problem  in 
the  western  United  States.   When  BMPs  are  implemented  the 
effect  of  forest  practices  on  sediment  production  is  low 
(Stednick,  1987) .   Determining  an  acceptable  level  or 
measurement  criteria  for  BMP  performance  for  controlling 
sediment  production,  is  necessary  for  assessing  results  and 
determining  if  adequate  protection  is  being  provided. 
Montana  has  not  established  what  this  level  is. 

It  should  be  remembered  that  BMPs  are  designed  only  to 
deal  with  sediment  production  from  surface  erosion.  In- 
channel  erosion  may  be  as  important  and  is  associated  with 
increases  in  water  yield  resulting  from  removal  of 
vegetation. 

This  section  will  examine  the  results  and  discuss  areas 
in  BMP  implementation  where  improvements  are  needed.   Again, 
"impacts"  are  defined  only  within  the  context  of  this 
qualitative  assessment  process. 
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Performance  in  the  Flathead  Basin 

Using  one  of  the  1988  Montana  EQC's  analysis 
techniques,  examining  the  percentages  of  total  practices  in 
each  of  the  5  rating  categories  (see  Table  6)  reveals  very 
high  levels  of  BMP  application  and  effectiveness  in  the 
Flathead  Basin.   Several  methods  of  analysis  (see  also 
Figures  6-9)  indicate  those  levels  are  substantially 
higher  than  the  levels  observed  in  the  1988  statewide  audits 
by  the  EQC.   However,  the  numbers  are  not  quite  so 
encouraging  considering  that  31  of  52  (59.6%)  of  the  timber 
sales  audited  in  the  Flathead  were  found  to  be  contributing 
at  least  one  minor  or  major  impact  to  water  quality.  Figures 
4  and  5  break  these  findings  down  by  ownership  and  between 
major  and  minor  impacts. 

Minor  impacts  and  their  accompanying  sediment 
production  should  not  be  underestimated.   If  land  managers 
are  to  control  the  cumulative  effects  of  their  practices 
then  they  should  strive  to  improve  upon  performance  in  this 
category,  as  well. 

Furthermore,  another  analysis  of  the  audit  data 
indicates  that  almost  1  major  impact  to  water  quality  is 
occurring  for  every  2  timber  sales  harvested  (Figure  9) . 
With  the  exception  of  the  non-industrial  private  group,  land 
owners  in  the  Flathead  Basin  performed  better  than  their 
counterparts  statewide. 
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One  might  ask  whether  the  approximate  90  %  level  of 
"application"  and  "effectiveness",  using  the  "%  of  total 
practices  audited"  analysis  technique,  means  that  adequate 
protection  of  water  quality  has  been  provided.   Such 
interpretations  rely  on  the  assumption  that  adequately 
applied  and  effective  practices  will  minimize  non-point 
source  pollution. 

It  was  found  that  only  one  practice  departure  on  a  sale 
could  produce  significant  sediment.   For  example,  consider  a 
stream  channel  that  has  been  severely  damaged  through  a 
reach  as  a  result  of  equipment  operation  in  and  around  it. 
There  is  only  one  practice  out  of  39  on  the  audit  form  that 
specifically  addresses  this  problem.   If  that  practice 
scored  the  lowest  it  possibly  could  on  the  scale,  and  no 
other  practices  were  found  to  be  contributing  impacts,  then 
the  lowest  that  timber  sale  would  score  would  be  a  97.4%  as 
the  level  of  protection  afforded  to  water  quality. 

This  implies  that  impacts  on  soil  and  water  resources 
can  be  occurring  on  a  timber  sale  even  though  the  percentage 
of  practices  that  were  adequately  applied  and  effective  is 
reported  as  high  as  97%.   In  that  context,  a  score  in  the  90 
percent  range,  could,  in  fact,  have  many  serious  impacts. 

Caution  should  be  exercised  in  relying  solely  upon 
interpretation  of  the  "%  of  Total  Practices  Audited"  as  a 
measure  of  protection  of  the  soil  and  water  resources.   It 
may  be  useful  in  determining  the  level  of  practice 
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application,  but  without  conducting  other  analyses  and 
examining  the  specific  practices  that  represent  the 
departures,  its  usefulness  for  assessing  the  level  of 
watershed  protection  is  limited.    This  position  is  based 
on  the  considerable  variation  in  the  "relative  importance" 
of  the  39  practices  audited  and  their  influence  in 
protecting  water  guality.   Therefore,  they  should  not  be 
weighted  egually  in  the  final  analysis.   Assigning  "weights" 
to  practices,  weighting  those  with  the  most  direct  influence 
on  sediment  production  higher  than  others,  and  recalculating 
a  weighted  percentage  of  practices  contributing  impacts 
would  potentially  be  a  more  meaningful  assessment  of 
performance. 

Targeted  levels  for  either  the  percentage  of  timber 
sales  contributing  impacts,  or  the  proportion  of  practices 
per  sale,  should  be  established  as  a  goal.   It  is  difficult 
to  establish  a  level  of  adeguacy  because  no  research  has 
been  conducted  to  date.   Clearly  more  can  be  done,  not  only 
to  establish  a  more  accurate  means  of  analyzing  practice 
performance,  but  more  can  be  done  on-the-ground  and  in  the 
planning  of  sales,  to  achieve  the  ultimate  goal  of  reducing 
the  number  of  impacts  resulting  from  land  management 
activities. 

A  conservative  goal  might  be  a  30%  reduction  in  each 
category  of  timber  sales  contributing  impacts.   This  would 
amount  to  lowering  the  current  level  of  sales  contributing 
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major  impacts  from  15  %  to  around  10  %.   On  sales  with  minor 
impacts,  this  would  amount  to  a  drop  from  60  %  to  about  40  % 
of  sales  producing  minor  impacts. 
Areas  For  BMP  Improvements 

Areas  where  specific  BMPs  were  found  lacking  included 
headwater  drainages,  sensitive  site  types,  riparian  areas 
with  inadeguate  width  SMZs  and  with  poorly  defined 
boundaries,  and  in  roads  drainage  and  maintenance  schedules. 
Improvements  are  clearly  needed  in  education,  sale  planning 
and  administration,  recognition  of  sensitive  site  conditions 
needing  protection,  improved  management  of  streamside 
vegetative  buffer  strips  (SMZs) ,  and  improved  construction 
and  maintenance  of  road  drainage  features  to  insure  that 
water  is  not  allowed  to  build  up  on  the  road  surfaces  or 
flow  directly  into  a  stream  without  passing  through  a 
vegetative  filter. 
1.   BMP  Education 

Education  in  BMP  application  and  effectiveness  is  one 
of  the  first  steps  toward  protecting  water  guality.   Various 
programs  have  existed,  on  a  limited  basis,  in  the  Flathead 
Basin  to  improve  BMP  education.   Efforts  include  education 
sessions  conducted  by  the  Montana  Logging  Association. 
Other  organized  efforts  include  the  Plum  Creek's 
distribution  of  the  State's  educational  BMP  handbook  to 
loggers  and  eguipment  operators.   These  efforts  are 
important  in  reaching  those  operators  whose  actions  are 
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often  times  the  most  critical  in  providing  protection  to 
soil  and  water  resources. 

While  it  is  widely  recognized  that  eguipment  operators 
can  create  damage,  it  was  observed  that  many  planners  and 
agency  personnel  were  lacking  a  general  familiarity  with  the 
terminology  and  practices  contained  in  the  State's  BMPs. 

The  U.S.Forest  Service's  Region  1  "Soil  and  Water 
Conservation  Practices  Handbook"  (U.S.D.A.  Forest  Service 
1988)  contains  many  of  the  same  practices,  but  specific 
subjects  such  as  the  management  of  an  streamside  management 
zone  (SMZ)  are  not  covered.    Practices  are  identified  for 
protection  of  the  stream  channel  under  the  sections 
addressing  timber  harvesting  and  transportation  planning  but 
specific  mention  of  maintaining  any  intact  vegetative  buffer 
zone  is  absent. 

A  basin-wide  organized  education  program  aimed  at 
promoting  BMPs,  similar  to  the  Montana  Logging  Associations 
efforts,  involving  loggers,  eguipment  operators,  resource 
planners,  and  administrators,  from  agencies  and  industry, 
would  be  of  value.   This  program  should  include  information 
on  identifying  areas  in  need  of  protection  or  special 
treatments . 
2.   Sale  Planning  and  Administration 

Adeguate  implementation  of  BMPs  is  not  solely  a  problem 
for  eguipment  operators  and  timber  sale  administrators. 
Freguently  their  actions  have  been  cited  as  those  most 
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directly  responsible  for  problems  on  the  ground.   In  some 
instances,  it  is  important  to  focus   attention  on  the 
planners  and  their  responsibility  for  road  and  sale  layout 
that  often  guides  the  administrator's  hand. 

On  some  Forest  Service  sales  the  administrator  felt  he 
was  restricted  by  sale  layout  and  road  design  in  the 
practices  that  he  could  carry  out.   He  felt  as  if  he  had 
been  "dealt  a  bad  hand"  and  was  in  the  position  of  making 
the  best  of  the  situation.   The  sale  layout  and  associated 
road  construction  may  play  a  large  part  in  the  skidding  and 
yarding  decisions  on  the  ground.   Some  latitude  to  make 
changes  in  sale  layout,  during  sale  administration,  was 
observed  to  recognize  needs  for  resource  protection. 
Education  of  the  sale  planners  and  active  participation 
between  administrator  and  planners  would  contribute  toward 
reducing  these  problems. 

The  work  of  sale  administrators  was  generally  very 
conscientious  but  varied  among  individuals.   No  one 
ownership  performed  any  better  than  another  in  sale 
administration.   Departures  in  BMPs  could  generally  be 
attributed  to  either  an  operator  not  paying  attention  to  the 
instructions  of  the  administrator  or  the  administrator's 
judgement  of  the  erosional  response  of  a  piece  of  ground  to 
the  prescribed  treatment.   Any  departures  seemed  to  be 
attributable  to  the  frequency  that  an  administrator  could 
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visit  a  site  to  supervise  and  the  administrator's  experience 
in  working  on  similar  site  conditions. 

The  involvement  of  soil  and  water  specialists  in 
situations  which  are  unfamiliar  to  an  administrator  is 
necessary  to  provide  inputs  on  the  risks  to  soil  and  water 
resources  associated  with  the  selection  of  a  harvesting 
system. 

The  following  suggestions  may  improve  the 
administrator's  success  in  BMP  implementation: 

1.  Increase  communication  with  sale  planners 
concerning  sale  layout. 

2.  Involve  soil  and  water  specialists  in  evaluation  of 
erosion  potential  and  BMP  selection. 

3 .  Employ  more  aggressive  flagging  of  streamside 
management  zones  (SMZ) . 

4 .  Increase  the  f reguency  of  on-site  practice 
inspection,  particularly  when  operations  are 
occurring  in  or  adjacent  to  sensitive  areas. 

5.  Conduct  post-sale  review  of  practices  and  their 
effectiveness.   This  might  provide  the  opportunity 
to  correct  any  departures  prior  to  an  impact 
occurring  while  eguipment  is  still  available 
on-site  or  in  an  adjacent  cutting  unit. 

6.  Develop  the  ability  to  interpret  the  need  for  BMPs 
and  their  correct  application  under  varying  site 
conditions. 
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3.   Recognition  of  Site  Conditions  in  Need  of  Protection 

On  sites  immediately  adjacent  to  perennial  streams, 
practices  were  generally  adeguate  or  better  in  application 
and  effectiveness.   Along  those  streams  that  had  easily 
recognized  beneficial  uses,  such  as  a  spawning  gravels  for 
fish,  the  planning  and  implementation  of  practices  was  very 
good  and  water  guality  was  adeguately  protected.   These 
sales  typically  had  the  SMZ  clearly  flagged  or  otherwise 
marked.   Topography  and  vegetation  were  used  to  establish 
boundaries  for  eguipment  operations. 

Problems  were  observed  more  freguently  in  headwaters 
areas  where  the  intermittent  and  ephemeral  channels  were 
weakly  defined  and  not  recognized  as  reguiring  protection. 
The  use  of  existing  roads  next  to  those  channels,  skidding 
of  logs  across  the  channels,   eguipment  operation  damaging 
riparian  vegetation,   and  ruts  from  eguipment  operation  were 
observed  on  a  number  of  sales. 

Teams  recognized  that  water  yield  increases  resulting 
from  harvest,  could  aggravate  erosion  in  these  channels  that 
freguently  do  not  flow  water.    Considerable  erosion  and 
off-site  delivery  of  sediment  was  observed  which  points  out 
the  need  for  recognizing  the  potential  impact  of  increased 
flows  in  stream  channels.   This  should  be  considered  this 
during  practice  selection. 

Low-gradient  roads,  constructed  in  fine-textured,  low- 
cohesion  soils  showed  a  considerable  amount  of  erosion 
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under  circumstances  that  would  not  generally  be  expected. 
Team  members  were  surprised  at  the  extent  of  erosion  taking 
place.   These  roads  were  low-standard,  without  drain 
features,  typically   3-4%  slope,  running  laterally  across 
the  base  of  the  sale  area  and  below  a  rather  large 
contributing  area  for  capturing  and  releasing  water. 
Discussions  centered  on  the  impacts  of  increased  water  yield 
resulting  from  the  harvest  or  raised  water  table  as  a 
possible  contributing  factor  to  increased  erosion  on  these 
sites. 

Other  considerations  include  available  water  on-site. 
Wet  sites,  which  either  have  relatively  shallow  ground  water 
tables,  poor  soil  drainage  characteristics,  higher 
precipitation,  or  a  combination  of  all  3,  were  a  source  of 
problems  observed  in  practice  application  and  associated 
impacts . 

Sales  audited  in  the  Swan  River  drainage  included  a 
proportionately  higher  number  of  wet  sites,  primarily 
associated  with  Plum  Creek  Timber's  ownerships.   Even  though 
attempts  had  been  made  to  delineate  SMZs,  avoid  equipment 
operation  in  those  areas,  and  operate  with  seasonal 
restrictions  such  as  winter  harvesting,  impacts  were  far 
more  frequent  on  these  sites  than  on  drier  sites.   On  wet 
sites  it  becomes  even  more  important  to  adjust  practices  to 
protect  the  integrity  of  the  SMZ  and  improve  the  frequency 
and  standards  of  road  surface  drainage. 
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On  sensitive  sites,  whether  due  to  soil  erodibility, 
slope,  or  excess  water,  the  following  practices  should  be 
given  added  attention: 

1.  Maintaining  an  adeguate-width  vegetation  buffer 
strip  (SMZ)  between  any  management  activity  and  the 
stream  channel.   Activities  presenting  potential 
soil  disturbances  include  construction  and 
maintenance  of  roads  and  eguipment  operation 
associated  with  harvesting  and  slash  treatment. 

2.  Improved  routing  of  water  from  road  surfaces  or 
skid  trails  through  an  adeguate  SMZ  to  slow  the 
flow  of  water  and  dissipated  it's  erosive  energy 
and  trap  any  sediment  being  transported. 

3.  Improved  freguency  of  road  maintenance  inspections, 
paying  particular  attention  to  berms  that  are 
retaining  water,  plugged  ditches  and  culverts  or 
surface  washing  caused  by  a  buildup  of  water 

While  each  of  these  practices  are  important  on  all 
sites,  they  become  particularly  critical  on  sensitive  sites. 
If  these  three  practices  are  implemented  with  increased 
freguency,  and  are  effective,  then  the  vast  majority  of 
potential  impacts  to  water  guality  will  be  avoided. 

It  becomes  more  important,  and  at  the  same  time  more 
difficult,  to  delineate  an  adeguate  SMZ  width  on  wet  site 
types.   The  ground  can  be  completely  saturated  beneath  thick 
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organic  layers  and  the  water  table  position  will  vary 
considerably  with  topography. 

Problems  were  observed  in  cases  where  large  tracts  of 
land  received  the  same  management  practices.   On  two 
different  sales,  teams  felt  that  the  impacts  that  were 
observed  were  simply  due  to  the  size  of  the  harvest  areas. 
The  areas  were  too  large  for  the  administrator  to  adequately 
control  the  practices  given  the  variation  in  site 
characteristics.   A  suggestion  was  made  to  possibly  break  up 
the  larger  unit  into  smaller  units  with  different  treatment 
zones.   Practices  should  be  altered  for  micro-site 
characteristics  such  as  wet  areas,  or  soils  texture  changes 
to  reduce  the  potential  for  impacts. 

On  smaller  parcels  of  land,  frequently  there  are  fewer 
miles  of  road  to  contend  with,  and  less  potential  for 
related  erosion  problems.   The  Department  of  State  Lands  had 
small  timber  sale  areas  and  frequently  timber  could  be 
reached  without  additional  road  construction.   This  limited 
the  opportunities  to  assess  new  road  construction  on  their 
lands,  but  resulted  in  fewer  impacts  from  their  sites. 
4.   Streams ide  Management  Zones 

The  SMZ  is  often  mistakenly  thought  of  as  a  zone  of 
"exclusion."   Timber  may  be  harvested  there,  but  additional 
care  must  be  exercised  in  these  "HIGH  HAZARD"  areas.   The 
1990  Montana  BMP  Work  Group  has  agreed  to  label  any 
harvesting  activity  in  an  SMZ  as  a  potential  "HIGH  HAZARD" 
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(Montana's  BMP  Work  Group,  1990).   Operations  in  these  areas 
may  significantly  increase  the  risk  of  damage  to  channels 
and  increased  delivery  of  sediment.   Any  harvest  activity 
should  include  special  considerations  to  minimize  soil 
disturbances,  and  maintain  a  zone  of  vegetation  which 
functions  to  provide  adeguate  channel  stabilization,  shade, 
sediment  filtering  capability  and  woody  debris  recruitment. 

In  cases  where  efforts  had  been  made  to  physically  mark 
an  SMZ,  the  practices  were  considerably  improved  over  those 
without  such  marking.   On  a  number  of  sites,  encroachment  of 
eguipment  into  stream  channels  or  excessive  disturbance  in 
the  SMZ  was  observed.   This  was  the  practice  with  the 
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highest  number  of  departures.   Improvements  are  needed  in 
the  recognition  and  identification  of  this  vegetative  buffer 
for  improved  protection. 

In  conjunction  with  this  study,  technigues  to  establish 
an  improved  method  for  delineation  of  the  SMZ  based  on 
vegetation  characteristics  and  slope  of  adjacent  land  were 
examined.   To  date,  the  only  accepted  distance  criteria  for 
SMZ  width  is  the  2  5  foot  minimum  width  established  by  the 
Environmental  Quality  Council  (EQC,  1989) .   The  Flathead 
Basin  audit  teams  were  in  general  agreement  that  the  natural 
topographic  slope  break  should  be  adhered  to  as  the  boundary 
for  the  SMZ  on  sites  with  deeply  incised  channels.   The  only 
exception  to  this  occurs  on  sites  where  the  slope  distance 
from  the  channel  to  the  slope  break  is  a  considerable  enough 
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distance  to  allow  the  establishment  of  an  adequate  buffer 
zone  on  the  lower  portion  of  the  slope. 

In  some  cases,  adequate  protection  of  the  SMZ  had  been 
provided  during  the  timber  harvest  operation,  only  to  be 
damaged  during  the  slash  reduction  and  site  preparation. 
This  points  out  the  need  for  protection  through  all  phases 
of  the  operation. 

Often  the  timber  that  was  left  in  an  adequate  SMZ  had 
been  lost  to  blowdown.   Shallow  rooting  systems,  typical  of 
timber  growing  over  high  water  tables,  produce  a  stand  which 
is  more  susceptible  to  wind  throw.   The  net  result  is  sudden 
exposure  of  mineral  soil,  often  either  directly  along  the 
stream  bank  or  on  side-slopes  where  efficient  transport  to 
the  channel  is  likely.   This  is  an  impact  which  was  observed 
and  is  well-recognized  in  the  profession. 

Attempts  to  get  professional  agreement  on  the  best 
approach  to  managing  SMZs  have  produced  no  widely  accepted 
solutions.   A  thorough  examination  of  alternative  practices 
and  treatments  to  minimize  the  effects  our  harvesting 
activities  is  needed. 
5.   Road  Drainage  and  Maintenance 

Together,  these  two  categories  represented  over  40%  of 
the  minor  impacts  and  over  30%  of  the  major  impacts  observed 
in  the  Flathead  audit.    The  most  common  problem  was  a  lack 
of  sufficient  cross  drains  to  prevent  the  buildup  of  water 
on  the  road  surface  and  in  the  ditches.   Another  common 
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problem  was  a  lack  of  drain  features  to  route  water  though 
an  SMZ  before  delivering  it  to  a  channel. 

Other  departures  commonly  observed  included  the  need 
for  improved  drainage  features  on  abandoned  or  closed  roads. 
There  was  disagreement  on  the  audit  teams  in  the  appropriate 
application  of  drain  features  on  these  roads.   On  some 
sites,  closed  roads  had  considerable  erosion  and  were 
lacking  any  features  to  get  the  excess  water  off  of  their 
surfaces.   Some  individuals  felt  this  was  in  accord  with  the 
language  of  the  practices  which  states,  "Leave  abandoned 
roads  in  a  condition  that  provides  adequate  drainage  without 
further  maintenance. ..." 

It  was  clear  that  different  standards  for  road  drainage 
were  being  considered  for  these  roads  by  some  team  members. 
Some  felt  that  erosion  of  the  surface  was  only  important  in 
the  context  of  future  travel  and  since  they  no  longer  had  to 
remain  passable  for  vehicle  travel,  erosion  of  the  surface 
was  not  a  problem.  Since  further  maintenance  would  not  be 
conducted,  their  drainage  was  adequate,  regardless  of 
whether  sediment  was  continuing  to  move  downslope. 

For  future  audits,  in  advance  of  the  field  work, 
agreement  should  be  reached  on  what  constitutes  an  abandoned 
road  and  what  are  the  appropriate  drainage  features  for 
these  roads. 

On  a  number  of  sales,  infrequent  inspection  and 
maintenance  of  ditches  and  culverts  led  to  impacts  to  water 
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quality.   Blocked  ditches  often  forced  water  onto  the  road 
surface  causing  erosion.   Blocked  culverts  forced  water  over 
road  surfaces  or.  backed  up  water  in  ditches,  potentially 
contributing  to  saturation  of  a  cut  slope,  its  subsequent 
failure,  and  further  blocking  the  pipe.   It  was  found  that 
appropriate  design  and  construction  can  be  negated  by 
inattention  to  the  maintenance  and  function  of  those 
features.   Routine  inspections  are  necessary  to  avoid 
problems  related  to  these  practices. 

These  road  related  practices  carry  considerably  more 
weight  in  terms  of  their  potential  impact  to  streams  and 
sediment  delivery  than  many  of  the  other  practices  audited. 
These  practices  typically  have  a  direct  relationship  to 
impacts,  particularly  in  the  locations  of  stream  crossings 
and  the  stretch  of  road  leading  up  to  and  away  from  the 
crossing. 

SUMMARY 

Although  it  is  true  that  forest  management  in  the 
Flathead  Basin  appears  to  be  performing  at  a  higher  level  of 
practice  application  and  effectiveness  than  in  the  rest  of 
Montana,  there  may  still  be  considerable  room  for 
improvement . 

It  is  unrealistic  to  expect  to  completely  avoid  impacts 
on  water  quality  from  timber  harvesting-related  activities. 
But,  it  was  demonstrated  that  in  many  cases  where  departures 
in  practices  had  occurred  and  impacts  were  observed  as  a 
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result,  very  little  additional  work  would  have  been  required 
to  correct  the  situation.   An  extra  water  bar  on  a  skid 
trail,  a  drain  dip  on  a  road,  an  extra  10  yards  of 
vegetative  buffer  along  a  channel,  or  a  cleaned  out  ditch  or 
culvert  were  the  simple  treatments  that  would  have  been 
necessary  to  eliminate  the  impact.   On  sales  where  SMZs  were 
damaged  it  would  have  required  very  little  additional  effort 
for  someone  to  walk  the  boundary  of  the  designated  zone  and 
tie  off  fluorescent  ribbon  as  a  means  of  delineating  areas 
for  protection.   Where  this  was  done,  there  was  a  noticeable 
improvement  in  protection. 

It  was  generally  recognized  that  when  BMPs  were 
"adequately  applied",  meaning  applied  in  such  a  way  that 
they  were  effective  in  "minimizing"  sediment  production,  an 
"adequate  level  of  protection"  was  also  noted  under  the 
effectiveness  rating.   However,  there  were  cases  in  which 
practices  were  rated  as  "adequately"  applied,  and  impacts 
were  still  observed.   The  reason  for  this  can  be  found  in 
the  definition  of  BMPs,  which  states,  "a  practice  . . . 
determined  to  be  the  most  effective,  practical  (including 
technological,  economic  and  institutional  considerations) 
means  of  preventing  or  reducing  . . .  pollution  generated  by 
non-point  sources..."  (40  CFR,  13  0. 2 (q) ) .   The  key  words 
here  are  "practical. .. .considerations. " 

A  good  example  to  illustrate  how  this  influences  the 
determination  of  an  "adequate  practice  application"  is  found 
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under  the  practices  associated  with  "Road  Planning  and 
Location".   If  pre-existing  roads,  located  along  stream 
channels,  were  used  during  a  timber  sale  they  would  be 
evaluated  for  "practice  application"  by  balancing  the  cost 
of  relocating  and  constructing  a  new  road  in  a  lower  risk 
location  against  the  potential  for  further  impacts  from 
continued  use  of  the  road.   If  teams  judged  the  costs  of  the 
new  road  and  the  alternative  yarding  practices  to  be 
prohibitive  and  the  amount  of  sediment  produced  to  be 
relatively  small,  then  the  location  of  the  road  would  be 
deemed  "adequate."   In  cases  such  as  this,  a  practice  could 
be  rated  as  "meeting  the  intent  of  BMPs  (rating  4)"  but 
score  "less  than  adequate  in  protecting  water  quality"  on 
the  effectiveness  scale.   Other  practices  such  as  "stable 
cut  slopes  in  the  road  prism",  and  "adequate  drainage 
features  on  closed  or  abandoned  roads"  present  similar 
problems. 

This  approach  represents  the  trade-offs  which  are 
presently  being  made,  as  each  management  activity  is  weighed 
against  the  impacts  that  it  produces  and  available 
alternatives  -  including  their  costs  and  practicality.   If 
no  practical  alternative,  short  of  eliminating  a  sale,  is 
recognized  then  practices  may  be  deemed  "adequately  applied" 
even  though  impacts  have  or  are  occurring. 

This  is  a  good  argument  against  the  adequacy  of  BMPs  as 
a  tool  to  monitor  the  level  of  protection  provided  a 
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watershed,  and  for  this  reason,  it  is  still  premature  to 
draw  the  conclusion  that  simply  because  BMPs  have  been  used, 
there  are  no  impacts  occurring  in  the  watershed. 

Other  concerns,  along  similar  lines,  include  the 
interpretation  of  "What  is  an  adequate  level  of  practice 
application  and  protection  on  sensitive  geologic  and 
topographic  site  conditions?"   This  begins  to  deal  with  the 
question  of  "erosional  risk"  and  how  management  alternatives 
and  specific  practices  must  be  altered  to  reduce  the 
potential  for  sediment  production.   What  is  adequate  on  some 
residual  soils  from  belt  rock  formations  has  been 
demonstrated  to  be  totally  inadequate  on  granitics  or 
alluvium.   Additional  guides  and/or  language  modification 
may  be  needed  in  the  BMPs  to  emphasize  this,  so  that  generic 
application  across  all  land  types  is  not  viewed  as  adequate. 

Although  the  information  exists  to  summarize  timber 
sales  by  erodible  landtypes,  it  was  found  that 
characterizations  of  an  entire  sale  as  "high"  to  "low" 
hazard  was  difficult  and  conditions  were  highly  variable  on- 
site.   Impacts  were  observed  in  what  were  characterized  as 
"low",  "moderate"  and  "high"  hazard  sites,  without  any 
distinct  pattern  of  occurrence.   More  importantly,  these 
impacts  were  not  so  much  a  reflection  of  how  the  entire  sale 
area  was  characterized,  but  instead  on  how  the  "higher" 
hazard  micro-sites  within  the  sale  boundary  were  evaluated 
for  appropriate  practice  selection  and  treated.   This 
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is  where  a  more  thorough  on-site  pre-sale  review,  revised 
BMPs  or  field  guides  for  correct  application  are  necessary 
to  avoid  problems. 

It  was  difficult  to  assess  what  the  real  impacts  to 
water  quality  were  from  the  silvicultural  treatments 
observed  on  these  52  timber  sales,  and  that  was  purely 
beyond  the  scope  of  this  project.   It  was  noted,  however, 
that  strictly  from  a  water  quality  standpoint,  the  surface 
erosion  in  the  clearcut  treatments  was  no  greater  than  that 
in  other  treatments.    Even  on  those  sites  with  slope 
steepness  of  greater  than  60%  it  appeared  that  very  little 
sediment  was  moving  downslope.   The  greatest  observable 
water  quality  risk  that  clearcutting  produced  was  in  the 
opening  of  the  canopy  and  the  associated  "blowdown"  of 
stands  adjacent  to  stream  channels.   This  introduced  a 
considerable  amount  of  sediment  into  the  channels  from  the 
root  masses  and  upset  the  stability  of  the  banks, 
undoubtedly  with  long-term  damage  and  impacts  to  water 
quality. 

It  seemed  that  road  location,  excessive  disturbance 
with  equipment  in  and  around  stream  channels  and  wet  areas, 
and  the  handling  of  water  from  skid  trails  and  abandoned 
roads  were  far  more  important  than  the  slope  steepness  or 
silvicultural  treatment. 

The  problems  observed  can  not  be  attributed  to  a  single 
ownership,  although  the  non-industrial  private  ownership 
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showed  consistently  higher  departures  and  impacts.   Nor  can 
problems  be  saddled  entirely  upon  the  administrators  and 
equipment  operators,  as  timber  sale  planning  has  been  shown 
to  contribute  as  well. 

Practices  in  the  Flathead  Basin  do  not  have  to  be 
drastically  changed  or  improved  upon  in  order  to 
significantly  improve  the  level  of  protection.   As  we  found, 
many  of  the  practices  are  already  being  applied.   What  is 
needed  is  an  improvement  in  the  "consistency"  with  which 
these  practices  are  applied.   This  can  be  achieved  by  not 
only  educating  the  professionals  on  the  ground,  but  by 
getting  management  to  rally  behind  these  practices,  take  an 
active  interest  in  applying  them,  and  support  their  people 
in  applying,  what  are  for  the  most  part,  cost-effective  land 
management  practices. 

At  a  time  when  the  timber  industry  in  Montana  is  facing 
increasing  pressure  from  the  public,  additional  attention  to 
these  practices  could  go  along  way  toward  improving  the 
working  relationship  between  those  groups  striving  for 
balanced  use  of  our  forested  lands. 

This  study  is  the  most  intensive  on-site  examination  of 
best  management  practices  for  forestry  ever  conducted  in 
Montana.   Future  assessments  of  this  type  will  serve  to  not 
only  measure  the  level  of  protection  being  implemented  to 
control  sediment  production,  but  will  also  evaluate  the 
performance  in  those  areas  where  this  report  indicates 
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improvements  are  necessary.   In  addition,  these  assessments 
will  provide  feedback  on  the  success  of  educational 
programs,  provide  the  continued  input  for  refinements  in  the 
BMPs  language  and  their  appropriate  application  under 
varying  site  conditions,  and  provide  resource  managers  the 
opportunity  to  interact  and  explore  those  forest  watershed 
relationships  which  are  influenced  by  their  practices. 
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